Motivation
A record of arrivals can be read in two ways. We may ask how many events have occurred by a specified time, or how long we must wait for a specified event. Counting and waiting describe the same sample path from different directions. Their probability distributions, however, live on different spaces: a count takes nonnegative integer values, while a continuous waiting time takes nonnegative real values. Keeping those objects distinct prevents several common modeling errors.
Imagine a processor checked at equally spaced clock ticks. At each tick a diagnostic packet arrives with probability , independently of the other ticks. This is a Bernoulli model. If packets can arrive at arbitrary real times, we need another clock. A homogeneous Poisson process supplies one when disjoint time intervals have independent counts and the statistical rule depends only on interval length. These are substantive modeling assumptions. A burst of packets generated by a shared trigger need not satisfy them, even when its average arrival rate appears constant.
These notes construct both models and explain the connection between them. Familiarity with independence, conditional probability, expectation and elementary integration is assumed. Throughout the unit, a process is a collection of random variables on one probability space; specifying its one-time distributions alone would leave dependence across time undetermined.
Definitions
Definition 1.1.1
(Bernoulli counting and waiting)
Let be independent random variables with and , where . Define
Then is a Bernoulli trial process, its counting process, and the tick of the th success. Write with for successive waiting times.
The convention for a geometric distribution matters: here counts the successful tick, so its support starts at . The count has binomial probabilities because a specified set of successful ticks has probability , and there are such sets. Counts on disjoint tick blocks are independent because they depend on disjoint collections of independent trials.
Definition 1.1.2
(Homogeneous Poisson process)
For , a homogeneous Poisson process starts at , has nondecreasing right-continuous integer-valued paths with jumps of size one, and has independent stationary increments satisfying
Its arrival times are , and its interarrival times are , with .
Stationary increments concern the distribution of a change over an interval, not the distribution of the total count. In fact grows with time, so this process is not stationary in the sense of time-invariant finite-dimensional distributions. The rate has units of events per time; an exponential density is , with the reciprocal time units required of a density.
Concept lensAnalytic
The Poisson law from short intervals
Instead of prescribing the Poisson law, one can prescribe only short intervals: independent stationary increments together with
Writing and splitting at gives and for , with . Solving these one at a time gives . In this description is the probability per unit time of one arrival in a very short interval.
Theorem/proposition
Proposition 1.1.3
(Geometric waiting and discrete memorylessness)
The waiting times are independent and identically distributed with
For integers , .
Theorem 1.1.4
(Exponential construction and arrival-time duality)
Take independent with density for . Put and , including . Then almost surely, this defines a homogeneous Poisson process, and
Conversely, Poisson-process interarrival times are independent exponential variables of rate .
The non-explosion assertion is essential: the maximum in the definition must remain finite at finite times. This is automatic for identical exponential rates, but it fails for some state-dependent clocks studied later. The density above is the Erlang distribution, a gamma distribution with integer shape and rate .
Proof sketch or proof idea
Proof
(Factorization over disjoint blocks of ticks)
For positive integers , the event specifying these first waiting times prescribes failures followed by a success in consecutive disjoint blocks. Its probability is
This factorization proves both the waiting-time law and independence. Also . Dividing two tail probabilities proves memorylessness. Summing the tail gives .
The continuous-time theorem needs three separate arguments: non-explosion, the Poisson law of a count, and the restart property behind independent increments.
Proof
(Exponential waits produce Poisson counts)
For an exponential variable, . Independence and the strong law give , proving non-explosion. To derive the count law directly, consider exactly ordered arrivals and no next arrival before . Multiplying the interarrival densities and the final survival probability gives the constant factor . Integrating over the ordered simplex, whose volume is , yields the Poisson mass. For , the same result is simply .
At a deterministic time , condition on the arrivals observed through . If the next waiting time has already lasted , memorylessness makes its remaining part exponential of the original rate, independently of that observed past. Subsequent full waiting times are independent by construction. Restarting at therefore produces a fresh copy of the process, proving stationary independent increments. Repeating this argument at arrival times gives the converse construction from a Poisson process. More formally, its first waiting time has tail , and independent increments plus right continuity give the same restart rule at each arrival time; one can justify passage from deterministic times by approximating the arrival from above with grid times.
Finally, the sum of exponential densities is obtained by induction using convolution. If the formula holds for , convolution with integrates from to , producing . The event identity follows directly from the monotonicity of the counting path and connects every waiting-time calculation to a count calculation.
Proof X-ray
Where equal rates enter the construction
The hypotheses enter at three distinct places. Independence of the lets the joint density of the first arrival times factor into interarrival densities, and equality of the rates is what collapses that product to , a constant on the ordered simplex . The count law is then only a volume computation, and the same constancy later makes the conditioned arrival times uniform order statistics. Memorylessness is used only at the restart: it turns the partly elapsed wait at time into a fresh exponential independent of the past, so every later interval reduces to a copy of an interval starting at , which is how the argument closes. Non-explosion is a separate input, supplied by the strong law.
Dropping equal rates breaks both steps. With rate for the th wait, is finite, so increases to a finite limit almost surely and for every beyond that limit. Even before explosion, the product of unequal densities depends on the positions , so the simplex integral no longer gives a Poisson mass.
What conditioning on a total changes
Independent increments are an unconditional statement about disjoint intervals. If we reveal the total count over their union, the component counts become dependent because they must add to that total. This distinction is often hidden by the phrase “arrivals are independent”, which does not identify the random variables or the information being conditioned on.
For , put and . Before conditioning, these are independent Poisson variables with means and . Dividing their joint probability by the probability that gives
The rate cancels: given the number of arrivals, the fraction of the interval determines the split. If the total is one, knowing that the first subinterval contains that arrival forces the second subinterval to contain none. Thus the two conditioned counts are dependent even though the original increments are independent. More generally, splitting into finitely many subintervals yields a multinomial distribution with probabilities equal to their relative lengths. The ordered arrival times, conditional on the total, have the distribution of the ordered values of independent uniform samples on the interval. This last statement follows from the constant ordered-arrival density used in the proof: after conditioning, the density on the simplex is .
Independent marking separates two clocks
Suppose each arrival receives a red label with probability , independently across arrivals and independently of the original process, with . Conditional on a total count , the red count is binomial with parameters . The unconditional joint probability of red and unlabeled arrivals on a length- interval is
The factorization proves independence of the two counts and identifies their Poisson laws. Applying the same argument simultaneously to disjoint time intervals proves that the two labeled counting processes are independent Poisson processes. The independent-label assumption is essential. A rule assigning the same color to every packet in a burst would not justify this factorization. Notice the change of information again: conditioning on the total produces dependent color counts, while leaving the total random produces independent color counts. A modeling argument must specify which of those experiments it describes.
Worked examples
Worked example 1.1.5
(Three successful diagnostic ticks)
Suppose . For the third diagnostic success to occur on tick , the first six ticks must contain two successes and tick must succeed. Hence
This is different from : the latter allows the third success on any of ticks through . Since , its expected value is . If the first four ticks failed, the conditional expected number of additional ticks until the first success is still ; the expected total tick of that success, conditional on those failures, is .
Worked example 1.1.6
(A future count after an observed past)
Packets follow a Poisson process of rate per minute. Even after observing ,
The future two-minute interval has mean count and is independent of the observed count. To find the chance that the second arrival occurs after minute , use the event identity:
A count event avoids an unnecessary integration of the Erlang density. The expected second arrival time is minute; an expected time does not imply probability one-half of exceeding that time.
Worked example 1.1.7
(A small-tick approximation and its limits)
Divide a time interval of length into ticks and independently place one event in each tick with probability , assuming . For fixed ,
Write the binomial coefficient as , cancel , and apply . This proves convergence of a single interval count. A full process convergence statement requires a joint-time argument and a chosen notion of path convergence; the calculation alone proves neither. Independent tick blocks do provide the corresponding joint limits for a fixed finite partition.
Common mistakes
Counterexample mode
Poisson marginals do not specify a process
A Poisson marginal law does not imply independent increments. To see the missing information concretely, take a single uniform random variable and define by the inverse distribution function of a Poisson law of mean , evaluated at that same for every . Each has the desired marginal distribution. Its increments are not independent. For , take and . The quantile intervals give
For the first equality, divide the interval length by . For the second, the overlap for both counts to equal one is , of length ; divide by . These conditional probabilities differ, proving dependence. The one-time count law cannot replace a process definition.
Two further errors concern what memorylessness and continuous time assert.
Common mistake
(Treating every waiting time as memoryless)
Memorylessness is a property of the geometric and exponential laws, not of waiting times in general. A deterministic wait of one minute fails it: after three-quarters of a minute, the remaining wait is one-quarter of a minute, not a fresh minute. Conversely, conditioning an exponential wait on survival does not make it deterministic; the remaining wait is again exponential with the same rate.
Common mistake
(Assigning probability to an exact arrival instant)
Because has a density, for every prescribed real . Arrival probabilities are computed over intervals, as , or by integrating ; a value of the density is not a probability.
Summary
Independent Bernoulli trials lead to binomial counts and independent geometric waits. Independent exponential waits lead to Poisson counts and a non-explosive continuous-time process. The identity is the bridge between counts and arrival times. Rate, independence, stationarity of increments and the support convention of a waiting law must all be stated before calculation.
Exercises
Quick check
(With , find and . Explain why the final tick is treated separately.)
The second success must fall on tick five itself.
Solution
There must be exactly one success in the first four ticks and a success on tick five. Thus . Independent waits give . Counting two successes anywhere in five ticks would include paths whose second success arrived earlier.
Quick check
(A Poisson process has rate per hour. Find the probability of no arrivals in the next half-hour after observing exactly five arrivals in the preceding hour. Find when time begins at zero.)
Use independent increments for the first part and the event identity for the second.
Solution · Answer
Independent stationary increments give , regardless of the preceding count. Also . Both formulas use interval lengths measured in hours.
Quick check
(Prove that for an exponential wait, and that . Explain what the second statement adds to the first.)
Expand to second order.
Solution
Taylor expansion gives . The count law gives . The first statement concerns at least one arrival; the second controls the chance of multiple arrivals in a short interval, which the first does not determine by itself.
Quick check
(Let two independent Poisson processes have rates and . Derive the law of their total count on a length- interval and state why the sum process has independent increments.)
Convolve the two Poisson mass functions and use the binomial theorem.
Solution
Independence and convolution give . Counts on disjoint intervals depend on disjoint increments of both independent input processes, so they are independent. Stationarity follows from dependence on interval length. Independent processes almost surely have no simultaneous arrival, so the sum has unit jumps and is Poisson of rate .