Motivation
An arrival counter treats every event as one unit. A data-transfer system often needs a different quantity: packets arrive at random times, and each carries a random number of bytes. The event count and the accumulated payload are different processes. Their expectations may both grow linearly, but their variances need different calculations because the number of payload terms is itself random.
Another distinction concerns the clock. Exponential waiting times produce a Poisson process, but repeated independent cycles need not be exponential. A renewal process retains the independent-cycle structure while allowing a general waiting law. It offers a useful bridge from Bernoulli and Poisson models to regenerative reasoning, while showing precisely which conclusions depend on memorylessness.
The arguments below use the strong law for independent integrable variables, elementary conditioning and nonnegative integration.
Definitions
Definition 1.2.1
(Renewal epochs, count and completed-cycle reward)
Let be independent identically distributed positive random variables, finite almost surely. Put , and
For independent identically distributed pairs , the completed-cycle reward is . Dependence within a pair is allowed: a long cycle may earn a large reward. Different pairs must be independent.
The sum defining the epochs diverges almost surely. Choose such that ; independent occurrences of these long-enough waits happen infinitely often, for instance by the strong law for their indicators. Thus finite times contain only finitely many renewals. If , then because the waits are positive. Rewards may be signed, provided their absolute expectation is finite.
Definition 1.2.2
(Marked and compound Poisson processes)
Let be a homogeneous Poisson process of rate . Assign independent identically distributed marks , independently of the entire arrival process. A finite-type marking uses probabilities summing to one. Its type counts are . A numerical marking gives the compound process
where the empty sum is zero. Numerical marks can be nonnegative payloads or signed changes; in the signed case need not be nondecreasing.
A mark attached to an event is distinct from a new event time. Marks equal to zero cause no visible jump of , even though the arrival counter jumps. Therefore one generally cannot recover the original event rate just by counting the nonzero jumps of the accumulated process.
Theorem/proposition
Theorem 1.2.3
(Renewal and renewal-reward strong laws)
If and , then almost surely
These are pathwise limits. They do not by themselves assert convergence of the corresponding expectations, and they do not describe reward earned inside an unfinished cycle.
Theorem 1.2.4
(Marking and compound laws)
For independent finite-type marks, the processes are independent Poisson processes with rates . For numerical marks, has stationary independent increments. For every real ,
If , then . If additionally , then .
Notice the second moment in the variance, not the variance of one mark. Randomness in the event count supplies the missing squared-mean term. Characteristic functions always exist because the exponential has absolute value one; a moment generating function would require an additional exponential-integrability assumption.
Proof sketch or proof idea
Inverting the clock and counting rewards
Proof
(Squeezing the count between epochs)
The strong law gives . Since and , divide by the appropriate indices and squeeze to obtain . The possibility at early times is irrelevant to this limit. For rewards, write
On the probability-one event where the reward strong law holds for the whole sequence, it holds along the random indices tending to infinity as well. No independence between and the rewards is needed. This is why dependence between a cycle's duration and its own reward is allowed.
Proof X-ray
Why the random index needs no independence
The proof makes two moves. The sandwich converts the strong law for epochs, a statement indexed by , into one indexed by continuous time: once ,
and both outer terms tend to . This requires , which holds because each is finite almost surely. The second move is pathwise: the reward averages converge on one probability-one event for the whole sequence, so they converge along any indices tending to infinity, random or not. Independence between and the rewards is never used, which is why a cycle's reward may depend on its own length. The hypothesis is used only to identify the limit; with the same sandwich gives .
There is also a finite-time equation. If is the waiting-time distribution and , conditioning on the first renewal gives
The integral is with respect to the waiting law, so it includes atoms and deterministic waits. To justify finiteness, use the earlier and let . By the time such waits have occurred, the accumulated time exceeds . The number of trials to obtain that many successes has finite mean and bounds above. Consequently is finite, and first-step conditioning is legitimate. This renewal equation does not assert that its solution is linear in time.
Separating labeled counts
Proof
(Factorizing the multinomial–Poisson mixture)
On an interval of length , condition on . The type counts have the multinomial law. For nonnegative integers summing to , multiplication by the Poisson probability gives
This factors into independent Poisson laws. For several disjoint intervals, the original increments are independent and the marks assigned to their disjoint sets of events are independent. Repeating the factorization proves independence of the entire type-count processes through their finite-dimensional distributions. This is more information than a statement about a single time.
Random sums and their fluctuations
Proof
(Conditioning a random sum on its count)
Given , independence of the marks gives . Averaging the Poisson series proves the characteristic-function formula. The same conditioning in disjoint intervals proves stationary independent increments, including when marks can be negative. If and are finite, conditional mean and variance are and . Total expectation and total variance yield
The total-variance identity follows by writing and observing that the cross term has zero conditional expectation. For the mean under only first absolute moments, absolute integrability follows from by nonnegative summation.
Concept lensAnalytic
Averages per event and averages per unit time
The mean mark is a reward per event; the arrival rate is events per unit time. Their product is the long-run reward rate. A compound Poisson variance includes two sources of randomness: the rewards themselves and how many rewards occur. Units help check the calculation, but only conditioning supplies the proof.
A finite-time distribution calculation for integer payloads
When numerical marks are nonnegative integers, their probability generating function offers more than moments. Write , and . Conditioning on the count gives
For the series converge absolutely. Differentiate there and compare coefficients in , obtaining
Only earlier probabilities appear on the right, so this is a concrete sequential method for a finite collection of payload probabilities. It is exact under the model, with no normal approximation. Analytic power-series differentiation inside the unit disk justifies coefficient comparison even when the mark mean is infinite; only finitely many terms enter each displayed coefficient. The initial probability is not generally , because zero marks may occur without changing the payload total.
As a check, suppose every mark is two. Then and the recurrence gives zero for every odd payload, while . This agrees with and shows that the accumulated payload need not itself have a Poisson distribution on all nonnegative integers. A random event count is Poisson; multiplying its jumps by two changes the attainable values.
There is also a useful two-time calculation. For and square-integrable marks, write . The bracketed increment is independent of , so
The total values are therefore generally dependent even though their disjoint increments are independent. This distinction is the same one encountered for ordinary Poisson counts, now with random jump sizes included.
A useful model check is the special case of zero-or-one marks. The nonzero events then form a thinned Poisson process, so the accumulated total really is Poisson, at the reduced rate. In contrast, a positive chance of marks greater than one introduces batch jumps and generally changes its distribution. A data table showing only total payloads cannot identify packet counts without information about the mark law. The event clock and the observation rule must both be specified before choosing a likelihood or interpreting an apparent arrival rate.
Worked examples
Worked example 1.2.5
(Payloads with two possible sizes)
Packets arrive at rate two per second. Independently, each contains one payload unit with probability and three units with probability . Then and . The accumulated payload has mean and variance after seconds. Using would omit the event-count fluctuations. The one-unit and three-unit packet counts are independent Poisson processes of rates and .
Worked example 1.2.6
(A renewal clock with no memorylessness)
Take deterministically and reward . Then and . Their long-run rates are and , matching the theorem. The process is not Poisson: between times and there can be no arrival, while a length- interval containing time two has one. Its increments are not stationary. Having the same mean waiting time as a rate- Poisson process does not make the clocks equivalent.
Worked example 1.2.7
(Dependent colors break the marking theorem)
Let a single independent fair coin choose one color for every arrival: all red or all blue. At a fixed time put . Then , , and . Thus for . Each color count has an atom produced by the common coin, and the counts are not independent. Each individual event is red with probability , but equal marking marginals do not supply independent marking across events.
Common mistakes
Counterexample mode
Rewards shared across cycles break the reward law
Claim: identically distributed rewards with finite mean always give long-run reward rate , so independence across cycles is a technicality. Take for every , let one fair coin choose , and set for all . Each has mean one, so the claim predicts rate one. But , hence , which equals or with probability one-half each and never equals one. The averages fail to converge to the mean because all rewards share one random source. The repair is the theorem's hypothesis: the pairs must be independent across cycles; dependence is allowed only within a pair.
Common mistake
(Expecting a fresh wait at a later observation time)
An ordinary renewal process starts with a fresh waiting time at time zero. Looking at it at a randomly chosen later time generally gives a different residual-life distribution. For deterministic waits of length two, the remaining wait depends on where the observation falls within the current cycle. Exponential memorylessness is a special property, not a consequence of independent successive full waits. This unit does not assert a general equilibrium residual-life theorem.
Common mistake
(Letting a finite arrival rate excuse an infinite-mean payload)
A reward law requires its integrability assumptions. A finite arrival rate does not make an infinite-mean nonnegative payload integrable: at a fixed positive time there is a positive chance of exactly one arrival, already enough to force infinite expected total payload.
Common mistake
(Passing a pathwise limit through an expectation)
A sample-path limit such as does not by itself give . Exchanging limit and expectation needs a bound, uniform integrability or a separate renewal theorem.
Summary
Renewal cycles separate event rate from the special exponential clock. The inverse strong law gives the event rate, and completed-cycle rewards give reward per time. Independent Poisson marking separates independent type processes. Compound sums retain Poisson event times but attach random jump sizes, adding event-count uncertainty to their variance.
Exercises
Quick check
(Independent renewal waits are one or three with equal probabilities. Each completed cycle earns its squared duration. Find the long-run event and reward rates.)
The reward of a cycle is a function of its own duration.
Solution · Answer
The mean wait is two. The mean reward is , even though reward and duration are dependent within each pair. The rates are event per time and reward units per time.
Quick check
(A rate-three Poisson process has independent marks zero and two with probabilities and . Find the mean and variance of and the rate of its visible jumps.)
Use , not , in the variance.
Solution
and , so mean and variance are and . A visible jump occurs only for a nonzero mark, hence its rate is , and its size is always two. Zero marks are still original arrivals.
Quick check
(A rate- Poisson process is independently divided into three labels with probabilities . Find the joint probability of counts at time .)
The three label processes are independent Poisson processes.
Solution · Answer
Independence of the type processes gives . The zero count in the second type supplies its exponential factor but no positive power.
Quick check
(For deterministic waits of length two, verify the renewal equation for , including the region .)
The waiting law is a unit atom at two.
Solution
The waiting distribution has unit mass at two. For , both and the integral are zero. For , the equation becomes , exactly the floor identity. The atom is included because the integration domain contains its right endpoint.