Motivation
A return probability can tend to zero while its sum diverges. This is why observing that a chain becomes unlikely to occupy one particular state does not decide whether it returns eventually. Transient and null-recurrent chains both have vanishing transition probabilities, but the total expected number of visits separates them. Positive recurrence has a different limiting structure, with periodicity determining which subsequences can converge.
This unit proves those pointwise statements rather than inferring them from a time average. A sequence can have average zero and still contain occasional spikes. The missing argument in the null-recurrent case is analytic: a Markov operator contracts a weighted square norm, forcing possible late-time values to become equal across states. An infinite invariant mass then forces that common value to vanish.
The prerequisite stationary criterion and aperiodic coupling theorem were proved in Countable recurrence, Kac's formula and coupling. Its proof uses return cycles and the strong law. We name and recall that dependency below. For the new vanishing argument we prove the needed weighted variance and subsequence facts explicitly. No renewal theorem, Tauberian theorem or unnamed analytic convergence theorem is assumed.
Definitions
Let be an irreducible stochastic matrix on countable . Write . A recurrent state has return probability one; it is positive recurrent when the mean return time is finite and null recurrent otherwise. A transient state has return probability below one. Irreducibility makes the classification common to the states.
Definition 2.8.1
(Invariant measure and weighted square norm)
An invariant measure is a vector with for each and . Its total mass may be infinite. For a real function define
Only functions of finite squared norm are used in the analytic argument. In particular the indicator of a single state belongs to this space because that state's invariant weight is finite.
Definition 2.8.2
(Period and cyclic classes)
A chain has period when the greatest common divisor of positive return lengths is . Choose a reference state . The cyclic class consists of states reachable from in a number of steps congruent to modulo , with indices read modulo . Well-definedness and the needed properties are proved below, rather than assumed from the notation.
Theorem/proposition
Theorem 2.8.3
(Vanishing outside positive recurrence)
For an irreducible countable chain that is transient or null recurrent,
The null-recurrent assertion does not require aperiodicity. In the transient case ; in the null-recurrent case the return sum diverges despite the individual terms tending to zero.
Theorem 2.8.4
(Periodic positive-recurrent limits)
Suppose instead the chain is positive recurrent with stationary probability and period . Its cyclic classes have . If , then for ,
In fact the entire law along this subsequence converges in total variation to the probability vector equal to on and zero elsewhere. Relabeling classes handles any initial state.
Proof sketch or proof idea
Transience and the recurrent-cycle measure
Proof
(Finite visits, and an invariant measure from return cycles)
If state has return probability , its total visits from , including time zero, have mean : the probability of at least visits is by the hitting-time Markov property. Starting elsewhere, first hit or never do so. Expected visits are the hit probability times this finite mean. Tonelli identifies expected visits with . Terms of a convergent nonnegative series tend to zero, proving the transient case.
For a recurrent chain fix and let be its first positive return. Define as expected visits to at times zero through . Then , each entry is positive and finite, and . Positivity follows from a simple path from to before returning. For finiteness at , a positive-probability simple path from to avoids another . Each visit to therefore has a fixed positive chance of ending further visits before hitting , giving a geometric upper tail.
Invariance follows by counting expected arrivals during the cycle instead of departures: initial and final states are both . All sums are nonnegative, so Tonelli applies even when total mass is infinite. Its total is . Thus the measure has infinite mass precisely in the null-recurrent case. This establishes all measure assumptions needed in the argument, not just a formal invariant equation.
A weighted-square-norm lemma
The null-recurrent case rests on the following lemma, stated first for aperiodic chains.
Lemma 2.8.5
(Vanishing under an infinite invariant measure)
Let be irreducible and aperiodic on countable , and let be an invariant measure with for every and . Then for all .
Proof
(Norm contraction, a variance identity and a flat limit)
Weighted Jensen, followed by invariance and nonnegative summation, gives
Every row sum defining is legitimate: its absolute expectation is bounded by the square-root of its second moment, and that row's second moment is bounded by the finite global squared norm divided by , because .
Fix a target state and set . Its values are transition probabilities, between zero and one. Its squared norms decrease to a finite limit. Consequently, for every fixed integer ,
The difference has an exact variance expression:
For a probability row, expanding the square proves the variance identity; its infinite version follows by square-integrability and nonnegative summation. Therefore if are both reachable in steps from a common state , their value difference tends to zero, since the displayed sum bounds a fixed positive multiple of that squared difference.
Aperiodicity supplies such a common length for every pair. At a reference state, finitely many return lengths already have greatest common divisor one. Concatenating return loops shows that return lengths are closed under addition. Their nonnegative combinations contain every sufficiently large integer: choose one length, obtain representatives for each residue modulo it from the other lengths, then add copies of the chosen length. In the finite residue group, nonnegative combinations generate inverses as well, so all residues occur. Append fixed paths from the reference state to and . For all sufficiently large common lengths both probabilities are positive. Thus for every pair.
Choose any subsequence along which converges to a scalar ; boundedness guarantees such a further subsequence. Every other coordinate then converges to the same . For any finite set ,
Increasing finite sets exhaust the infinite invariant mass. Therefore . Every subsequential scalar limit is zero, so the whole sequence tends to zero at every coordinate. This is the full pointwise conclusion, not merely a Cesàro statement.
Proof X-ray
Why infinite invariant mass forces zero
The proof turns an inequality into an identity and then an identity into a limit, and each hypothesis has one job.
- Invariance is used twice: in the contraction , through , and in the variance identity, where is invariant for . Finiteness of makes the starting norm finite.
- The key move is the variance identity. The norms are decreasing and bounded below, so the drop over any fixed steps tends to zero; the identity converts that vanishing drop into asymptotic flatness of on pairs of states reachable from a common state in steps.
- Aperiodicity supplies such a common length for every pair, so flatness spreads to all of .
- Infinite mass is used only at the end. A flat profile with value has squared norm about on a finite set , which stays below only if .
The argument closes by compactness: every subsequence has a further subsequence with a flat limit, every such limit is zero, and so the whole sequence tends to zero pointwise. This supplies the missing step from averages to pointwise vanishing without a renewal or Tauberian black box.
Applying the lemma to the cycle measure proves the aperiodic null-recurrent case. The hypothesis of infinite mass cannot be dropped, as a two-state chain shows.
Counterexample mode
Finite invariant mass leaves a flat nonzero limit
Claim: for an irreducible aperiodic chain with any invariant measure, . Take and every entry of equal to . Then is invariant with finite mass , and for , . Every step of the proof goes through: is constant, so the norm drop is zero, and is exactly flat with . The last step fails, because does not force . So the claim is false. The repaired statement needs ; with finite mass the flat limit is the stationary probability , here , which is the positive-recurrent limit.
Cyclic classes remove the period obstruction
Proof
(Cyclic classes, mass balance and coupling on each class)
Two path lengths from to the same state differ by a multiple of : append a fixed path back to and compare two return lengths. Hence the class labels are well-defined. Every transition advances a label by one, and all classes occur along a return path. The restriction of to any class is irreducible, since paths between states of that class have length divisible by . Its period is one because return lengths divided by have greatest common divisor one.
For any invariant measure, nonnegative summation and the one-class advance give
If total mass is infinite, every class therefore has infinite mass, since there are finitely many classes. The restricted measure is invariant for . Apply the weighted-norm lemma within each class. To pass from multiples of to , first take the steps and average the vanishing probabilities from the resulting states; bounded convergence for a probability-weighted countable sum justifies this average. This proves the null-recurrent assertion for every period.
For a stationary probability, the same mass identity instead gives . The stationary probability for the restricted chain is on that class. The earlier aperiodic positive-recurrent coupling theorem now applies to this irreducible aperiodic restriction. Its proof was explicit: the independent product chain has invariant probability, is irreducible by the common-length path argument, hence is recurrent by the cycle criterion; it reaches a fixed diagonal state almost surely, after which the two copies use the same transitions. The probability of not yet meeting bounds total variation. This recalled argument uses the previously proved stationary criterion, not an external renewal limit.
Starting from , the law after steps is a probability mixture on . Each starting state in that mixture converges under to there; total-variation distances are bounded by one, so averaging preserves convergence. States in other classes have probability zero at those times. This proves the cyclic limits and completes both theorems.
Worked examples
Worked example 2.8.6
(A transient biased walk)
For nearest-neighbor walk on all integers with right probability , the return probabilities are zero at odd times and at time . The binomial power series gives expected visits to zero equal to two. Thus the chain is transient, and every transition probability tends to zero. Finiteness here is a statement about the entire visit sum, stronger than the individual-term limit.
Worked example 2.8.7
(Recurrent returns that still vanish)
Simple symmetric integer walk has return generating function . As increases to one this diverges, so nonnegative monotone convergence makes the return sum infinite: the chain is recurrent. Counting measure is invariant with infinite mass. The period is two, and restricted to either parity class is irreducible and aperiodic, with infinite counting mass. The proved lemma gives vanishing transition probabilities. No stationary probability exists, since the stationary recurrence makes its coordinates affine on the entire integer line, with no nonzero nonnegative summable solution. This is null recurrence, not transience.
Worked example 2.8.8
(A periodic limit with unequal state weights)
Let move to with probability and to with probability ; both return to surely. Classes are and , with period two and stationary vector . From , every even-time law is the point mass at , and every odd-time law puts on . These are exactly twice the stationary weights on the compatible class, with zero on the other class. The factor two is necessary to make each subsequence's limiting vector have mass one.
Concept lensAlgebraic
The period seen in the spectrum
For this finite chain the cyclic limits are a spectral decomposition. The matrix has eigenvalues , with right eigenvectors and and left eigenvectors and for and , normalized so that . Hence, for ,
The second term is the alternation between the two classes. For any irreducible finite chain of period , the Perron–Frobenius theorem says that the eigenvalues of modulus one are exactly the th roots of unity, each simple.
Common mistakes
Invariant measure and invariant probability must remain distinct. The infinite mass used in the proof is a reason for a zero pointwise limit, not an object that can be normalized by writing an infinite denominator.
Common mistake
(Reading vanishing terms as a finite visit sum)
Vanishing probabilities do not imply that the total number of visits has finite mean. A divergent nonnegative series can have terms tending to zero: the symmetric integer walk has , yet its return sum is infinite and the walk is recurrent. Transience is a statement about the sum, not about the terms.
Common mistake
(Upgrading a Cesàro limit to an ordinary limit)
Cesàro convergence does not establish an ordinary limit without an additional argument. In the periodic worked example above, the averages of converge to while the terms alternate between and . In the null case the weighted variance proof supplies the missing argument.
Common mistake
(Taking a periodic limit without a residue class)
In a periodic positive-recurrent chain, taking the limit of without specifying a residue class mixes incompatible supports and misses the factor . The correct limit along is on the compatible class and zero elsewhere.
Summary
Transient vanishing follows from finite total visits. Null-recurrent vanishing follows from invariant-measure contraction and the impossibility of a nonzero flat finite-norm limit on infinite mass. Cyclic classes handle periodicity; positive-recurrent subsequences converge to the stationary probability normalized within the compatible class.
Exercises
Quick check
(A transient state has return probability , and the chance of hitting it from is . Find expected total visits from and explain why its transition probabilities vanish.)
Count visits from the state itself first.
Solution · Answer
Mean visits from the state itself are . Multiplying by the hit probability gives . This finite mean equals the nonnegative transition-probability sum, whose terms must tend to zero.
Quick check
(For a deterministic three-state directed cycle, find the stationary probability and all three residue-class limits from one fixed state.)
Each cyclic class is a single state.
Solution
Stationarity gives and the period is three. At times congruent to zero, one and two, the laws are point masses at the corresponding successive cycle states. They equal three times the stationary weights on each singleton cyclic class.
Quick check
(Suppose a sequence has squared norms at most under an infinite measure with finite positive entries, and along a subsequence all coordinates tend to one scalar . Prove . Why does finite total mass change the conclusion?)
Work on finite sets first.
Solution
For each finite , finite-sum convergence gives . Exhaustion makes arbitrarily large, forcing . With finite total mass, a nonzero constant has finite squared norm, so the same bound cannot force it to vanish.
Quick check
(Use symmetric integer walk to explain, with a return generating function, why a pointwise transition limit zero does not imply transience.)
Let increase to one.
Solution
The generating function is , diverging as , so the expected return count is infinite and the walk is recurrent. The infinite-measure argument still gives each transition probability tending to zero. The series sum and the limit of its terms answer different questions.