Motivation
Sampling at jump times counts visits. Sampling at a fixed real time counts how long visits last. The two samples need not even admit probability normalizations together. A chain can have no stationary probability for its embedded visits while still having one for its real-time occupation, because fast clocks make long discrete excursions short in real time.
The familiar instruction “divide by the departure rate and normalize” therefore needs hypotheses and a proof. What visit measure is being divided? Is the resulting total finite? Does the local rate matrix define a process at every finite time? And what happens when the average stationary departure rate is infinite? This note answers those questions through complete return cycles, rather than treating a formal balance equation as sufficient.
The argument uses the countable discrete stationary criterion, the strong law for independent integrable variables and the minimal jump construction; each use is identified below.
Definitions
Let be countable with at least two states. Assume finite strictly positive departure rates and an irreducible stochastic matrix with . Define the stable conservative rate matrix
The embedded chain follows . Conditional on its visited states, its holding times are independent exponentials of means , obtained using fresh independent unit exponentials. We consider the minimal pure-jump construction and assume it is non-explosive from every initial state. Stationarity means for every , not just .
The weights that convert visits into time are defined from one return cycle of the embedded chain.
Definition 3.5.1
(Cycle visit measure and real-time weights)
If is recurrent, fix and let be its first positive return in jump steps. Define its cycle visit measure and real-time weights by
The reference visit is included, the terminal return excluded, and . A continuous return cycle starts on entry into , includes its initial exponential hold, and ends on the next entry into . It is not the infimum of positive times at , which would be zero during the initial holding interval.
Theorem/proposition
Theorem 3.5.2
(General stationary occupation criterion)
Under the stated irreducibility, positive finite rates and non-explosion assumptions, a stationary probability exists exactly when the embedded chain is recurrent and . It is unique and
Every recurrent embedded chain is itself enough to guarantee non-explosion of this construction, even when its discrete mean return length is infinite. Recurrence with gives no stationary probability and an infinite mean continuous cycle.
Theorem 3.5.3
(Normalized visit law and stationary departure weights)
For recurrent irreducible , any strictly positive finite-entry invariant measure is a scalar multiple of . If has stationary probability , then a stationary continuous-time probability exists exactly when , and
Whenever a stationary real-time law exists, its average departure weight is
If , then is the embedded stationary probability. If , that normalization is unavailable; the embedded chain is null recurrent, although the real-time law is stationary.
Proof sketch or proof idea
Visit cycles and their invariant measure
Proof
(Cycle measures, uniqueness and non-explosion)
Recurrence makes every embedded return cycle finite almost surely. Each is positive by a simple path reaching before returning to . It is finite because, from each visit to , a fixed positive-probability simple path hits without another . The visit count therefore has a geometric upper tail. Counting expected arrivals instead of departures over a complete cycle gives : its initial and final states are both . Nonnegative summation justifies this identity even when .
For later normalization, we also prove uniqueness among finite-entry invariant measures. Let with . Iterating this equation while excluding intermediate visits to gives : each finite collection of paths departing from and not yet returning contributes a nonnegative part, and increasing the collection gives the cycle sum. At equality holds since . The difference is nonnegative and invariant. Subtraction is valid coordinatewise because both incoming sums are finite at each target. If any , a positive-probability path from to would force , a contradiction. Thus .
Conditional on the embedded path, expected holding time in each visit to is . Tonelli therefore gives expected real-time occupation in one continuous cycle and mean total length , allowing infinity. Successive cycles are independent and identically distributed because the destination and exponential draws restart independently at each return. They include independent holds at of mean . Those holds alone have divergent cumulative sum by the strong law. Thus recurrent embedded paths cannot explode. From any other state the first embedded hit of is almost surely finite, contributing only finitely many finite holds.
Proof X-ray
How the cycle measure forces uniqueness
Three facts carry the uniqueness half. First, any invariant dominates : unrolling along paths that leave and have not yet returned produces exactly the excursion sums defining , plus a nonnegative remainder that is discarded. Second, the two measures agree at because . Third, irreducibility transports any excess: the nonnegative invariant difference satisfies , so excess at would appear at . Recurrence is used only to make invariant; finiteness of the entries of is what makes the subtraction legitimate.
Finite mean occupation constructs an actual stationary law
Proof
(Time averages give an invariant probability)
Suppose . Apply the independent integrable strong law to continuous cycle lengths and their time rewards at each state. Completed cycles give occupation proportion . The unfinished cycle has no effect: if cycle completion times are , then and , while any unfinished reward is bounded by that cycle's length. A finite initial path to likewise has no effect. Hence from every state, time-averaged occupation tends almost surely to .
We must still prove that this probability vector is preserved by the actual transition semigroup. Set . Bounded convergence of pathwise occupation gives . These are probability vectors and the limit has total one. Pointwise convergence therefore implies total-variation convergence: on a large finite set the coordinate differences vanish, and the complementary masses are small because the limiting vector is a probability.
For fixed , the Markov property gives
Its difference from is at most by comparing the two integration intervals. Meanwhile total-variation convergence permits passing the probability vector through the bounded kernel . The two limits give . Thus invariance is established for the semigroup, without an unsupported interchange of an unbounded generator and an infinite sum.
Proof X-ray
Why the time average is invariant
The sufficiency proof never solves ; it constructs as a limit and checks invariance for each kernel .
- Embedded recurrence supplies independent identically distributed cycles, and is exactly the integrability needed for the strong law. Without it the time fractions still exist but tend to zero.
- Non-explosion is used through the Markov property with stochastic rows, which turns the shifted average into .
- The key move is the window comparison: shifting the averaging window by changes the average by at most .
- The upgrade from pointwise to total-variation convergence, possible because the limit is a probability vector, is what allows the limit to pass through the bounded kernel .
The argument closes because both sides of converge, to and respectively.
Necessity and uniqueness
Proof
(A stationary law forces recurrence and finite occupation)
Suppose instead that a stationary real-time probability exists. It has positive mass at every state, because a finite positive-rate path can be followed within any positive time with positive probability. For any fixed grid spacing , the discrete skeleton is irreducible for the same reason and has stationary law . The countable discrete stationary criterion, proved in the earlier recurrence unit, makes that skeleton recurrent. It visits at arbitrarily late times. Since every individual hold at is finite almost surely, this requires infinitely many genuine returns in the embedded chain. Thus is recurrent.
If the continuous cycle mean were infinite, their completed count per unit time would tend to zero. To see this without applying an integrable strong law to infinite-mean lengths, truncate each length at a fixed level, apply the ordinary strong law, then let the level increase; the truncated means tend to infinity. Time at is bounded above by the sum of one-more-than-completed independent exponential holds there. Their sample mean tends to , so the time proportion at tends to zero. A finite initial path changes no limit.
For stationary initial law, however,
The proportions are bounded by one. Bounded convergence would force their expectations to zero, a contradiction. Hence . The already-proved occupation limits from every starting state, averaged under , then identify . This proves necessity and uniqueness.
Finally, . Its total is finite exactly when the embedded cycle has finite expected jump length, equivalently positive recurrence of . Normalizing in that case gives its stationary probability. If that total is infinite, the recurrent embedded chain is null recurrent and its visit measure cannot be normalized, even though dividing by rates has produced a finite time measure. The rate balance equation follows coordinatewise as a consequence, since for each target; it was not the existence proof.
The infinite-measure case of the second theorem rests on a distinction between almost sure finiteness and finite expectation.
Concept lensStructural
A finite path can have an infinite expected jump count
Non-explosion says each bounded time interval contains finitely many jumps almost surely. It does not impose a finite expectation on every random jump count or a finite value of . Cycle occupation, not an assumed finite global jump intensity, is what supports the theorem's infinite-measure case.
Worked examples
Worked example 3.5.4
(A nonreversible three-state cycle)
Let the embedded chain move deterministically , with departure rates . Its cycle visit measure is , but holding weights are . Thus , the stationary real-time vector is , and mean continuous return cycle from zero is . The embedded stationary vector is uniform. The average departure weight is , and multiplying the real-time weights by rates then normalizing recovers that uniform vector. No detailed balance or reversibility is required.
Worked example 3.5.5
(Null-recurrent visits with positive-recurrent time)
Take embedded simple symmetric walk on the integers, whose recurrence and infinite counting invariant measure were proved in the transition-limit unit. Choose . Recurrence ensures non-explosion through repeated independent holds at zero. With ,
The real-time chain has stationary probability and finite mean zero-cycle , while the embedded chain has no stationary probability. Indeed . Faster clocks far from zero compress excursions that have infinitely many expected jump steps into finite expected time.
Worked example 3.5.6
(Positive-recurrent visits slowed into null recurrence)
Let and, for , let the chain move up with probability and down with probability . Its invariant cycle measure is , ; edge balance verifies this, and its finite total four proves embedded positive recurrence. Now choose . Rates are bounded, so the continuous process is non-explosive. But and at every positive state, making . The embedded chain has stationary probability, but the real-time chain is recurrent with infinite mean cycle and no stationary probability. Slow distant states reverse the normalization outcome.
Common mistakes
Common mistake
(Dividing by a zero or virtual departure rate)
An absorbing state has zero departure rate and lies outside the theorem's positive-rate irreducible setting. Dividing its stationary mass by zero is meaningless; absorbing classes must be handled separately. Virtual self-updates also differ from genuine jumps, which is why is stated.
Common mistake
(Normalizing the visits before dividing by rates)
Normalizing a visit probability before dividing by rates is convenient only when that visit probability exists. The null-recurrent example requires an infinite visit measure. Conversely, a finite visit normalization does not guarantee a finite time normalization, as the third example shows.
A formal solution of is not the path argument above; the theorem explicitly controls recurrence, lifetime and the finite total time weight. The following chain shows that this control cannot be skipped.
Counterexample mode
A probability solving the balance equations need not be stationary
Claim: a probability vector with is stationary. Take the birth–death rates for and for , and . Then on every edge, so . The embedded chain moves up with probability at positive states and is transient, and the expected lifetime from zero is , so the minimal process explodes. For , Markov's inequality gives , hence the total mass of is at most and . The theorem excludes this chain twice: the embedded chain is transient, and non-explosion fails.
Summary
Return-cycle visit weights become time weights after division by positive departure rates. Embedded recurrence makes cycles and non-explosion meaningful; finite total time weight is exactly the stationary-probability criterion. Finite average departure weight recovers an embedded stationary probability, while infinite departure weight allows stationary real time with null-recurrent jump visits.
Exercises
Quick check
(An embedded two-state alternation has departure rates two and five. Find the real-time equilibrium, the mean zero-to-zero cycle and the average departure weight.)
Both cycle visit counts equal one.
Solution · Answer
Cycle visits are and time weights , so and . The mean cycle is and the average departure weight is .
Exercise 3.5.7
(Use the reflected biased walk of the third example, but set . Find , the real-time stationary weights and the average departure weight.)
Core
Solution(3)
- 1
Divide the cycle visits by the rates
With , and , the time weights are and for .
- 2
Sum the geometric series
, and . Rates are bounded and the embedded chain is recurrent, so the process is non-explosive.
Because: .
- 3
Compute the departure weight
The embedded cycle mass is , so , which is finite.
Quick check
(Multiply every departure rate by while leaving unchanged. Explain the effect on stationarity, the mean cycle, non-explosion and the average departure weight.)
The visit measure depends only on .
Solution · Answer
Holding times and divide by , while the normalized time weights and the embedded chain stay unchanged. Non-explosion is preserved by a deterministic change of time scale. The average departure weight multiplies by ; an infinite value remains infinite.
Exercise 3.5.8
(For the embedded symmetric integer walk and rates with , classify the real-time stationary normalization and decide whether its average departure weight can be finite.)
Core
Solution(3)
- 1
Non-explosion comes from recurrence
The construction defines a process at every finite time for each .
Because: The embedded walk is recurrent, and the theorem gives non-explosion for every choice of finite positive rates.
- 2
Compute the total time weight
Key step
With the counting visit measure , , which is finite exactly when . For the chain is positive recurrent in real time with ; for the mean continuous cycle is infinite and there is no stationary probability.
- 3
Test the departure weight
Whenever the normalization exists, for every integer , so . The average departure weight is never finite.