Skip to content

MATH4240

Stochastic Processes

Bernoulli and Poisson counting processes, discrete- and continuous-time Markov chains, Poisson splitting and superposition, and Brownian motion.

5 chapters · 23 sectionsEN · 繁 · 简
My course notes

Your progress

0/ 23 sections marked complete

Next section: 1.1 Bernoulli trials and Poisson clocks

Completion marks are saved in this browser.

This course in the web

In the MATH4240 notes

Theorem cards
55
Definitions
31
Counterexamples
23
Concept lenses
17
Histories of ideas
0
Proof X-rays
26

No concept lens from this course is recorded in the web yet.

Open this course in the knowledge map

Counts cover the 23 public sections of this course and what is written in them, not recorded connections. 0 of these sections are indexed into the web in detail so far.

In the webTheorem cards 55 · Definitions 31 · Counterexamples 23 · Concept lenses 17 · Proof X-rays 26

No concept lens from this course is recorded in the web yet.

Open this course in the knowledge map

Counts cover the 23 public sections of this course and what is written in them, not recorded connections. 0 of these sections are indexed into the web in detail so far.

Chapter 1 · 4 sections

Counting processes

Trials, counts and arrival clocks.

Theorems 8 · Definitions 7Counterexamples 4Lenses 4Proof X-rays 4

Chapter outline →See this chapter on the web →Not yet indexed in the web

  1. 1.1

    Bernoulli trials and Poisson clocks

    Construct Bernoulli and Poisson counting processes, prove waiting-time laws, and distinguish increments from arrival times.

  2. 1.2

    Renewal rewards and compound Poisson processes

    Derive renewal and reward rates, independent marking, compound-Poisson transforms and moments, and an exact integer-payload recursion.

  3. 1.5

    Nonhomogeneous Poisson processes and the cumulative clock

    Construct a deterministic time-varying arrival model, prove its interval and waiting laws, and handle quiet intervals and finite observation horizons carefully.

  4. 1.6

    Equilibrium renewal and the life remaining

    Derive the age and residual-life law from cycle rewards, construct a stationary renewal process, and distinguish time sampling from a fixed observation time.

Chapter 2 · 10 sections

Discrete Markov chains

Hitting, return cycles and finite-state equilibrium.

Theorems 23 · Definitions 15Counterexamples 10Lenses 5Proof X-rays 10

Chapter outline →See this chapter on the web →Not yet indexed in the web

  1. 2.1

    Returns, recurrence and equilibrium

    Derive first-step equations and visit-count recurrence, then separate finite-chain stationarity, return means and convergence.

  2. 2.2

    Countable recurrence, Kac's formula and coupling

    Prove the countable stationary criterion, Kac's return formula, regenerative occupation averages and aperiodic total-variation convergence by coupling.

  3. 2.3

    Martingales, bounded stopping, and gambler's ruin

    Prove bounded stopping and finite-interval hitting laws, and examine an unbounded stopping counterexample.

  4. 2.6

    Optional extension: survival conditioning and quasi-stationary laws

    Prove a finite positive-block survival limit, calculate a quasi-stationary law, and distinguish surviving populations from individual paths.

  5. 2.7

    Finite absorbing blocks and occupation

    Classify closed classes before computing the fundamental matrix, entrance probabilities, occupation costs and absorption-time moments.

  6. 2.8

    Transient, null and periodic transition limits

    Prove vanishing transition probabilities by a weighted-square-norm argument, then derive cyclic limits for periodic positive recurrent chains.

  7. 2.9

    Random walks between absorbing boundaries

    Solve hitting probabilities and expected durations for a biased random walk, and examine the limit of a distant boundary.

  8. 2.10

    Stationary time reversal and reversibility

    Reverse a finite Markov chain in equilibrium, test detailed balance, and distinguish stationarity from reversible motion.

  9. 2.11

    The strong Markov property and restart times

    Prove the restart rule for discrete-time chains and distinguish stopping times from choices that look into the future.

  10. 2.12

    Branching by generations and extinction

    Use offspring generating functions to determine extinction probabilities and distinguish critical and supercritical behavior.

Chapter 3 · 7 sections

Continuous-time chains

Holding clocks, uniformization and birth–death balance.

Theorems 17 · Definitions 7Counterexamples 7Lenses 7Proof X-rays 10

Chapter outline →See this chapter on the web →Not yet indexed in the web

  1. 3.1

    Generators, jump clocks and birth–death chains

    Build jump chains with exponential clocks, prove bounded-rate uniformization, and derive birth–death equilibrium with an explicit explosion counterexample.

  2. 3.2

    Minimal jump processes and non-explosion

    Construct minimal jump paths, prove a proper-function non-explosion test and pure-birth lifetime criterion, and control killed finite approximations.

  3. 3.3

    Birth–death resistance and equilibrium

    Derive resistance hitting probabilities, classify recurrence and stationary occupation, and distinguish summable formal weights from non-explosive equilibrium.

  4. 3.4

    Absorbing birth–death chains and extinction

    Derive extinction and first-passage formulas from finite-boundary scale and Green functions, then analyze linear branching in all three regimes.

  5. 3.5

    Embedded visits and stationary time

    Prove the general countable-chain conversion from visit-cycle measures to holding-time equilibrium, without assuming a finite stationary jump rate.

  6. 3.6

    Finite uniformization and certified transition laws

    Construct a finite CTMC with an independent Poisson clock, prove its semigroup and invariant-law identities, and bound the error in a finite transition-law calculation.

  7. 3.7

    The M/M/1 queue: a busy period and its load

    Use birth–death first passage to derive the busy-period transform, its finiteness and the stationary queue length.

Chapter 4 · 1 section

4. Poisson splitting and superposition

Prove splitting and superposition and examine conditional dependence.

Theorems 2 · Definitions 1Counterexamples 1Proof X-rays 1

Chapter outline →See this chapter on the web →Not yet indexed in the web

  1. 4.1

    Splitting and combining Poisson processes

    Prove independent Poisson thinning and distinguish conditional counts from independent streams.

Chapter 5 · 1 section

Brownian motion

The continuous-time limit of the random walk: definition, martingales, the reflection principle and hitting times.

Theorems 5 · Definitions 1Counterexamples 1Lenses 1Proof X-rays 1

Chapter outline →See this chapter on the web →Not yet indexed in the web

  1. 5.1

    Brownian motion and the reflection principle

    Define standard Brownian motion, derive its covariance, scaling and martingales, and use the reflection principle to find the law of the running maximum and of hitting times.

Vercel visit statistics and public-page performance measurements each require separate consent. Neither is needed to use the website.

Website function storage (used when needed)

Security, sign-in and your chosen preferences use storage when needed. This is separate from optional statistics.

Withdrawing stops future statistics. The page reloads if analytics code needs unloading. It does not delete data already sent.

Read Cookie and storage details