MATH4240
Stochastic Processes
Bernoulli and Poisson counting processes, discrete- and continuous-time Markov chains, Poisson splitting and superposition, and Brownian motion.
Your progress
0/ 23 sections marked complete
Next section: 1.1 Bernoulli trials and Poisson clocks
Completion marks are saved in this browser.
This course in the web
In the MATH4240 notes
- Theorem cards
- 55
- Definitions
- 31
- Counterexamples
- 23
- Concept lenses
- 17
- Histories of ideas
- 0
- Proof X-rays
- 26
No concept lens from this course is recorded in the web yet.
Counts cover the 23 public sections of this course and what is written in them, not recorded connections. 0 of these sections are indexed into the web in detail so far.
In the webTheorem cards 55 · Definitions 31 · Counterexamples 23 · Concept lenses 17 · Proof X-rays 26
No concept lens from this course is recorded in the web yet.
Counts cover the 23 public sections of this course and what is written in them, not recorded connections. 0 of these sections are indexed into the web in detail so far.
Chapter 1 · 4 sections
Counting processes
Trials, counts and arrival clocks.
Theorems 8 · Definitions 7Counterexamples 4Lenses 4Proof X-rays 4
Chapter outline →See this chapter on the web →Not yet indexed in the web
- 1.1
Bernoulli trials and Poisson clocks
Construct Bernoulli and Poisson counting processes, prove waiting-time laws, and distinguish increments from arrival times.
- 1.2
Renewal rewards and compound Poisson processes
Derive renewal and reward rates, independent marking, compound-Poisson transforms and moments, and an exact integer-payload recursion.
- 1.5
Nonhomogeneous Poisson processes and the cumulative clock
Construct a deterministic time-varying arrival model, prove its interval and waiting laws, and handle quiet intervals and finite observation horizons carefully.
- 1.6
Equilibrium renewal and the life remaining
Derive the age and residual-life law from cycle rewards, construct a stationary renewal process, and distinguish time sampling from a fixed observation time.
Chapter 2 · 10 sections
Discrete Markov chains
Hitting, return cycles and finite-state equilibrium.
Theorems 23 · Definitions 15Counterexamples 10Lenses 5Proof X-rays 10
Chapter outline →See this chapter on the web →Not yet indexed in the web
- 2.1
Returns, recurrence and equilibrium
Derive first-step equations and visit-count recurrence, then separate finite-chain stationarity, return means and convergence.
- 2.2
Countable recurrence, Kac's formula and coupling
Prove the countable stationary criterion, Kac's return formula, regenerative occupation averages and aperiodic total-variation convergence by coupling.
- 2.3
Martingales, bounded stopping, and gambler's ruin
Prove bounded stopping and finite-interval hitting laws, and examine an unbounded stopping counterexample.
- 2.6
Optional extension: survival conditioning and quasi-stationary laws
Prove a finite positive-block survival limit, calculate a quasi-stationary law, and distinguish surviving populations from individual paths.
- 2.7
Finite absorbing blocks and occupation
Classify closed classes before computing the fundamental matrix, entrance probabilities, occupation costs and absorption-time moments.
- 2.8
Transient, null and periodic transition limits
Prove vanishing transition probabilities by a weighted-square-norm argument, then derive cyclic limits for periodic positive recurrent chains.
- 2.9
Random walks between absorbing boundaries
Solve hitting probabilities and expected durations for a biased random walk, and examine the limit of a distant boundary.
- 2.10
Stationary time reversal and reversibility
Reverse a finite Markov chain in equilibrium, test detailed balance, and distinguish stationarity from reversible motion.
- 2.11
The strong Markov property and restart times
Prove the restart rule for discrete-time chains and distinguish stopping times from choices that look into the future.
- 2.12
Branching by generations and extinction
Use offspring generating functions to determine extinction probabilities and distinguish critical and supercritical behavior.
Chapter 3 · 7 sections
Continuous-time chains
Holding clocks, uniformization and birth–death balance.
Theorems 17 · Definitions 7Counterexamples 7Lenses 7Proof X-rays 10
Chapter outline →See this chapter on the web →Not yet indexed in the web
- 3.1
Generators, jump clocks and birth–death chains
Build jump chains with exponential clocks, prove bounded-rate uniformization, and derive birth–death equilibrium with an explicit explosion counterexample.
- 3.2
Minimal jump processes and non-explosion
Construct minimal jump paths, prove a proper-function non-explosion test and pure-birth lifetime criterion, and control killed finite approximations.
- 3.3
Birth–death resistance and equilibrium
Derive resistance hitting probabilities, classify recurrence and stationary occupation, and distinguish summable formal weights from non-explosive equilibrium.
- 3.4
Absorbing birth–death chains and extinction
Derive extinction and first-passage formulas from finite-boundary scale and Green functions, then analyze linear branching in all three regimes.
- 3.5
Embedded visits and stationary time
Prove the general countable-chain conversion from visit-cycle measures to holding-time equilibrium, without assuming a finite stationary jump rate.
- 3.6
Finite uniformization and certified transition laws
Construct a finite CTMC with an independent Poisson clock, prove its semigroup and invariant-law identities, and bound the error in a finite transition-law calculation.
- 3.7
The M/M/1 queue: a busy period and its load
Use birth–death first passage to derive the busy-period transform, its finiteness and the stationary queue length.
Chapter 4 · 1 section
4. Poisson splitting and superposition
Prove splitting and superposition and examine conditional dependence.
Theorems 2 · Definitions 1Counterexamples 1Proof X-rays 1
Chapter outline →See this chapter on the web →Not yet indexed in the web
- 4.1
Splitting and combining Poisson processes
Prove independent Poisson thinning and distinguish conditional counts from independent streams.
Chapter 5 · 1 section
Brownian motion
The continuous-time limit of the random walk: definition, martingales, the reflection principle and hitting times.
Theorems 5 · Definitions 1Counterexamples 1Lenses 1Proof X-rays 1
Chapter outline →See this chapter on the web →Not yet indexed in the web
- 5.1
Brownian motion and the reflection principle
Define standard Brownian motion, derive its covariance, scaling and martingales, and use the reflection principle to find the law of the running maximum and of hitting times.