Why remove the variance assumption?
A sample can have a stable long-run average even when its variance is infinite. The finite-variance proof in the earlier convergence note cannot be applied directly in this situation: its variance bound may be infinite. Truncation lets us discard exceptional observations only finitely often along almost every path, while retaining enough variance control for the remaining observations.
Statement and preparation
Theorem 18.1.1
(Integrable iid strong law)
If are independent and identically distributed real random variables with and , then
No finite second moment is assumed.
Proof
(Truncation changes only finitely many terms)
Put . The tail-integral formula and monotonicity of give
The first Borel–Cantelli lemma therefore shows that for all sufficiently large , almost surely. On each such path the two partial sums differ eventually by a fixed finite number, so their averages have the same limit. Independence was not needed for this truncation step.
A summable variance bound
Proof
(Summable variances after truncation)
We next show . By identical distribution and Tonelli's theorem for nonnegative terms,
For the inequality, if then ; if , the integral comparison applies. Since variance is at most the second moment, the claim follows. Each is bounded and the variables remain independent, although they are no longer identically distributed.
Convergence of the weighted centered series
Proof
(Almost-sure convergence of the centered series)
Set . These independent variables have mean zero and . We prove that converges almost surely. Choose increasing integers so that . Kolmogorov's maximal inequality for independent centered finite-variance variables gives, for every ,
Taking the increasing union over preserves that bound. Its sum over is finite, so Borel–Cantelli implies that, almost surely, for every sufficiently large all partial sums beginning at have absolute value at most . For any , their intervening sum is the difference of two such partial sums and has magnitude at most . The Cauchy criterion proves convergence. The maximal inequality is the one proved in the earlier convergence note; it is essential that the are independent and centered.
Kronecker's lemma and completion of the proof
We need the following elementary lemma.
Lemma 18.1.2
(Kronecker's lemma)
If converges, then .
Proof
(Summation by parts)
Write and . Summation by parts yields
If , the Cesàro mean on the right also tends to , proving the lemma.
The lemma now converts the convergent weighted series into convergence of averages.
Proof
(Completion of the strong law)
Apply Kronecker's lemma pathwise with . The previous section gives almost surely. Moreover, dominated convergence gives , because . Their Cesàro means also tend to . Thus the averages of converge almost surely to , and the finite-discrepancy argument transfers the result to .
Proof X-ray
How the four steps share the hypotheses
The proof is a chain of four reductions, and each hypothesis is consumed at an identified link. Integrability is used twice in the same form: the tail sum makes truncation change only finitely many terms, and the bound makes the truncated variances summable. Identical distribution is what lets both estimates be written with the single law of . Independence is used once, in Kolmogorov's maximal inequality for the centered variables ; the truncation step needs only the first Borel–Cantelli lemma, and Kronecker's lemma is deterministic.
The non-obvious choice is a truncation level that grows with the index. Unless is bounded, a fixed level would change infinitely many terms, while no truncation can leave the variances infinite. The level balances the two series, and the division by in matches it, so that Kronecker's lemma converts convergence of back into averages. The centering constants cause no harm, because they converge to by dominated convergence.
Worked examples and a boundary
Worked example 18.1.3
(A finite mean with infinite variance)
Let have density for , and zero elsewhere. Direct integration gives but . For iid copies the theorem gives almost surely. This conclusion does not follow from the earlier finite-variance proof, although the theorem above applies.
Worked example 18.1.4
(Truncation estimates for a tail of order )
For the law of the previous example, the tail is for . The truncation series is , so almost surely only finitely many terms are changed. For the variance step,
which grows without bound. Nevertheless . Finally . Each ingredient of the proof is visible here: the truncated second moments are unbounded, but they grow slowly enough that division by makes them summable.
Worked example 18.1.5
(Without a finite mean the averages cannot settle)
Let be iid with , with no restriction on signs. Since
the series diverges. The events are independent, so the second Borel–Cantelli lemma gives infinitely often, almost surely. If on some path converged to a finite limit , then
which is impossible when infinitely often. Hence, almost surely, the averages have no finite limit. For iid averages, integrability is therefore also necessary for a finite almost-sure limit; the counterexample below shows one way the averages can diverge.
Counterexample mode
An infinite mean can destroy a finite limit
Let for , and take iid copies. Here . For every positive integer , the bounded variables satisfy the strong law, with mean . Taking the countable intersection of their probability-one events gives
Consequently the averages tend to almost surely. This is one example, not a converse asserting a particular limit for every nonintegrable signed distribution.
Common mistakes
Common mistake
(Truncation must change only finitely many terms)
A single unusually large term need not invalidate the strong law; what matters is that the events on which the chosen truncation changes a term have summable probabilities. Truncating every term at one fixed level fails this test whenever : the events are independent with constant probability, so by the second Borel–Cantelli lemma infinitely many terms are changed.
Common mistake
(Vanishing variances do not make a series converge)
Convergence of is stronger than merely having ; the proof uses the sum of the variances and a maximal inequality.
Common mistake
(A finite mean requires )
Finite mean means for a real-valued random variable: cancellation of divergent positive and negative expectations does not define .
Exercises
Quick check
(Let for . Find , decide whether is finite, and state the iid sample-mean limit.)
Solution · Answer
The tail formula gives . The second moment diverges, since . The sample mean converges to almost surely by the integrable iid theorem.
Quick check
(For iid integrable , prove that almost surely using Borel–Cantelli.)
Solution · Answer
For each , monotonicity and the tail-integral formula imply . Borel–Cantelli gives eventually almost surely. Intersecting these events for , , yields the result.
Quick check
(Suppose and . Use the lemma to find the limit of ; explain why convergence of the unweighted series is not needed.)
Solution · Answer
The alternating harmonic series converges, so Kronecker's lemma gives limit . The unweighted partial sums alternate between and and do not converge; the lemma assumes convergence only after division by .
Quick check
(Let for all , where . Does the strong-law conclusion hold with ? Identify the missing hypothesis.)
Solution · Answer
Every sample average equals , which is never , so it does not converge to the mean. The variables have the same marginal distribution but are not independent. Identical marginals alone do not supply the theorem's independence.