Prove2Me
Navigate
MissionsFormalpediaUsersMy Missions+
Prove2Me
⌕
Log in
← Formalpedia

Finite Markov chains: transition matrices, stationarity, irreducibility, period, reversibility

Definition
mm_basic

by Shuze Chen · Aug 21, 2026 · Mathlib c5ea003 (Lean v4.30.0)

markov-chainsmixing-timesprobability

This file sets up the basic vocabulary of finite Markov chains, following Chapters 1-2 of Levin-Peres-Wilmer. A Markov chain on a finite state space VVV is presented by its transition matrix: a matrix P∈RV×VP\in\mathbb R^{V\times V}P∈RV×V with nonnegative entries whose rows sum to one. A probability distribution on VVV is a nonnegative vector of total mass one, acting on the right of PPP as a row vector, and a distribution π\piπ is stationary when it is fixed by one step of the chain, πP=π\pi P=\piπP=π.

The chain is irreducible if for every pair of states x,yx,yx,y there is a ttt with Pt(x,y)>0P^t(x,y)>0Pt(x,y)>0.

The period of a state xxx is the greatest common divisor of its return-time set T(x)={t≥1:Pt(x,x)>0}\mathcal T(x)=\{t\ge1:P^t(x,x)>0\}T(x)={t≥1:Pt(x,x)>0}, formalized as the largest natural number dividing every element of T(x)\mathcal T(x)T(x) (a state with empty return set receives the junk value 000).

The chain is aperiodic when every state has period one. The file further introduces: the lazy version 12(I+P)\tfrac12(I+P)21​(I+P) of a chain;

harmonic functions, those hhh with h(x)=∑yP(x,y) h(y)h(x)=\sum_y P(x,y)\,h(y)h(x)=∑y​P(x,y)h(y) at every state;

the detailed balance equations π(x)P(x,y)=π(y)P(y,x)\pi(x)P(x,y)=\pi(y)P(y,x)π(x)P(x,y)=π(y)P(y,x), whose validity makes the chain reversible;

the time reversal P^(x,y)=π(y)P(y,x)/π(x)\hat P(x,y)=\pi(y)P(y,x)/\pi(x)P^(x,y)=π(y)P(y,x)/π(x), whose rows vanish where π(x)=0\pi(x)=0π(x)=0 since division is total;

simple random walk on a graph, which from xxx moves to a uniformly chosen neighbour of xxx; the uniform distribution on VVV;

and the random walk on a finite group GGG with increment distribution μ\muμ, which steps from aaa to hahaha with probability μ(h)\mu(h)μ(h) -- equivalently, its transition probability from aaa to bbb is μ(ba−1)\mu(ba^{-1})μ(ba−1).

Definition code
import Mathlib.Data.Matrix.Basic
import Mathlib.Data.Real.Basic
import Mathlib.Data.Nat.Lattice
import Mathlib.Combinatorics.SimpleGraph.DegreeSum

/-!
Basic vocabulary of finite Markov chains, following Levin–Peres–Wilmer,
*Markov Chains and Mixing Times*, Chapter 1.

A chain on a finite state space `V` is presented by its transition matrix
`P : Matrix V V ℝ`, acting on the right of row vectors: a distribution `μ`
evolves to `μ ᵥ* P` in one step, and the `t`-step transition probabilities
are the entries of `P ^ t`.
-/

namespace MarkovMixing

noncomputable section

open scoped BigOperators

variable {V : Type*} [Fintype V] [DecidableEq V]

/-- A matrix is **stochastic** (a transition matrix) if its entries are
nonnegative and every row sums to `1` (LPW §1.1, Eq. (1.1)). -/
def IsStochastic (P : Matrix V V ℝ) : Prop :=
  (∀ x y, 0 ≤ P x y) ∧ ∀ x, ∑ y, P x y = 1

/-- A **probability distribution** on the finite state space `V`, presented as a
row vector: nonnegative entries summing to `1`. -/
def IsDist (μ : V → ℝ) : Prop :=
  (∀ x, 0 ≤ μ x) ∧ ∑ x, μ x = 1

/-- A distribution `π` is **stationary** for `P` if `π = π P` (LPW §1.1, Eq. (1.4)). -/
def IsStationary (P : Matrix V V ℝ) (π : V → ℝ) : Prop :=
  IsDist π ∧ Matrix.vecMul π P = π

/-- A chain is **irreducible** if any state can reach any other: for all
`x y` there is a `t` with `P^t(x,y) > 0` (LPW §1.3). -/
def Irreducible (P : Matrix V V ℝ) : Prop :=
  ∀ x y : V, ∃ t : ℕ, 0 < (P ^ t) x y

/-- `T(x) = {t ≥ 1 : P^t(x,x) > 0}`, the set of possible return times to `x`
(LPW §1.3). -/
def returnSet (P : Matrix V V ℝ) (x : V) : Set ℕ :=
  {t : ℕ | 1 ≤ t ∧ 0 < (P ^ t) x x}

/-- The **period** of a state `x` is `gcd T(x)` (LPW §1.3).  It is formalized as
the largest natural number dividing every element of `returnSet P x` (which is
the gcd whenever `T(x) ≠ ∅`, and junk-value `0` when `T(x) = ∅`). -/
def period (P : Matrix V V ℝ) (x : V) : ℕ :=
  sSup {d : ℕ | ∀ t ∈ returnSet P x, d ∣ t}

/-- A chain is **aperiodic** if every state has period `1` (LPW §1.3). -/
def Aperiodic (P : Matrix V V ℝ) : Prop :=
  ∀ x : V, period P x = 1

/-- The **lazy version** `Q = (I + P)/2` of a chain (LPW §1.3). -/
def lazy (P : Matrix V V ℝ) : Matrix V V ℝ :=
  (2⁻¹ : ℝ) • (1 : Matrix V V ℝ) + (2⁻¹ : ℝ) • P

/-- A function `h` is **harmonic** for `P` (at every state) if
`h(x) = ∑_y P(x,y) h(y)` (LPW §1.5.4, Eq. (1.28)). -/
def Harmonic (P : Matrix V V ℝ) (h : V → ℝ) : Prop :=
  ∀ x : V, h x = ∑ y, P x y * h y

/-- `π` and `P` are in **detailed balance** if `π(x) P(x,y) = π(y) P(y,x)` for
all `x, y`; a chain admitting such a distribution is called reversible
(LPW §1.6, Eq. (1.30)). -/
def DetailedBalance (P : Matrix V V ℝ) (π : V → ℝ) : Prop :=
  ∀ x y : V, π x * P x y = π y * P y x

/-- The **time reversal** `P̂(x,y) = π(y) P(y,x) / π(x)` of a chain with
stationary distribution `π` (LPW §1.6, Eq. (1.33)). -/
def timeReversal (P : Matrix V V ℝ) (π : V → ℝ) : Matrix V V ℝ :=
  fun x y => π y * P y x / π x

/-- **Simple random walk** on a graph `G`: from `x`, move to a uniformly chosen
neighbor of `x` (LPW §1.4, Eq. (1.13)). -/
def graphWalk (G : SimpleGraph V) [DecidableRel G.Adj] : Matrix V V ℝ :=
  fun x y => if G.Adj x y then ((G.degree x : ℝ))⁻¹ else 0

/-- The **uniform distribution** on a finite state space. -/
def uniformDist (V : Type*) [Fintype V] : V → ℝ :=
  fun _ => (Fintype.card V : ℝ)⁻¹

/-- The **random walk on a finite group** `G` with increment distribution `μ`:
from `a`, move to `h * a` where `h ∼ μ`, so the transition probability from
`a` to `b` is `μ (b * a⁻¹)` (LPW §2.6). -/
def groupWalk {G : Type*} [Group G] [Fintype G] (μ : G → ℝ) : Matrix G G ℝ :=
  fun a b => μ (b * a⁻¹)

end

end MarkovMixing
Source
D. A. Levin, Y. Peres, E. L. Wilmer, Markov Chains and Mixing Times, AMS 2009, https://documents.epfl.ch/groups/i/ip/ipg/www/2013-2014/Random_Walks/markovmixing.pdf, Ch. 1 (Sections 1.1-1.6) and Section 2.6
Read-back

What the Lean code literally says, in plain math · claude-fable-5

Read-back: Def_mm_basic (namespace MarkovMixing)

Throughout, VVV is an arbitrary type assumed finite (with decidable equality), and all matrices are V×VV \times VV×V matrices with real entries, viewed as functions of two arguments: P x yP\,x\,yPxy is written P(x,y)P(x,y)P(x,y) below. Sums ∑x\sum_x∑x​ range over all elements of the finite type VVV.

IsStochastic. For a real matrix PPP indexed by VVV, the predicate IsStochastic PPP asserts the conjunction of two conditions: every entry is nonnegative, ∀x,y∈V, 0≤P(x,y)\forall x, y \in V,\ 0 \le P(x,y)∀x,y∈V, 0≤P(x,y); and every row sums to one, ∀x∈V, ∑y∈VP(x,y)=1\forall x \in V,\ \sum_{y \in V} P(x,y) = 1∀x∈V, ∑y∈V​P(x,y)=1. (Rows, not columns: the sum for fixed first index xxx runs over the second index yyy.) If VVV is empty, both conditions hold vacuously.

IsDist. For a function μ:V→R\mu : V \to \mathbb{R}μ:V→R, the predicate IsDist μ\muμ asserts that μ(x)≥0\mu(x) \ge 0μ(x)≥0 for every x∈Vx \in Vx∈V and that ∑x∈Vμ(x)=1\sum_{x \in V} \mu(x) = 1∑x∈V​μ(x)=1. If VVV is empty the sum is 000, so no μ\muμ on an empty type satisfies this.

IsStationary. For a real matrix PPP indexed by VVV and a function π:V→R\pi : V \to \mathbb{R}π:V→R, the predicate IsStationary P πP\ \piP π asserts that π\piπ satisfies IsDist (entrywise nonnegative with total mass 111, as above) and that the row vector π\piπ multiplied on the right by PPP equals π\piπ; componentwise, ∑x∈Vπ(x) P(x,y)=π(y)\sum_{x \in V} \pi(x)\, P(x,y) = \pi(y)∑x∈V​π(x)P(x,y)=π(y) for every y∈Vy \in Vy∈V. No stochasticity assumption on PPP is part of this definition.

Irreducible. For a real matrix PPP indexed by VVV, the predicate Irreducible PPP asserts: for every pair of states x,y∈Vx, y \in Vx,y∈V there exists a natural number t≥0t \ge 0t≥0 such that the (x,y)(x,y)(x,y) entry of the matrix power PtP^tPt is strictly positive, 0<(Pt)(x,y)0 < (P^t)(x,y)0<(Pt)(x,y). The exponent t=0t = 0t=0 is allowed; since P0P^0P0 is the identity matrix, the case x=yx = yx=y is always witnessed by t=0t = 0t=0 regardless of PPP, so the definition only imposes a genuine condition on pairs x≠yx \ne yx=y.

returnSet. For a real matrix PPP indexed by VVV and a state x∈Vx \in Vx∈V, returnSet P xP\ xP x is the set of natural numbers

R(P,x)={ t∈N∣t≥1 and 0<(Pt)(x,x) },\mathcal{R}(P,x) = \{\, t \in \mathbb{N} \mid t \ge 1 \ \text{and}\ 0 < (P^t)(x,x) \,\},R(P,x)={t∈N∣t≥1 and 0<(Pt)(x,x)},

i.e. the set of strictly positive times ttt at which the diagonal entry of PtP^tPt at xxx is strictly positive. Time t=0t = 0t=0 is explicitly excluded.

period. For a real matrix PPP indexed by VVV and a state x∈Vx \in Vx∈V, period P xP\ xP x is defined as the supremum, taken in the natural numbers, of the set of common divisors of the return set:

period⁡(P,x)=sup⁡ { d∈N∣d divides every t∈R(P,x) },\operatorname{period}(P,x) = \sup\,\{\, d \in \mathbb{N} \mid d \text{ divides every } t \in \mathcal{R}(P,x) \,\},period(P,x)=sup{d∈N∣d divides every t∈R(P,x)},

where R(P,x)\mathcal{R}(P,x)R(P,x) is the returnSet just described. This supremum is Lean's supremum on N\mathbb{N}N, which returns 000 for a set that is empty or unbounded. Consequently: if R(P,x)\mathcal{R}(P,x)R(P,x) is nonempty, the common divisors form a bounded set and the value is the greatest common divisor of all return times; but if R(P,x)\mathcal{R}(P,x)R(P,x) is empty (the state never returns), then every natural number vacuously divides every element, the divisor set is all of N\mathbb{N}N, which is unbounded, and the defined period is 000, not ∞\infty∞.

Aperiodic. For a real matrix PPP indexed by VVV, the predicate Aperiodic PPP asserts that period⁡(P,x)=1\operatorname{period}(P,x) = 1period(P,x)=1 for every state x∈Vx \in Vx∈V, with period as defined above (in particular, a state with empty return set has period 000 and so witnesses failure of this predicate).

lazy. For a real matrix PPP indexed by VVV, lazy PPP is the matrix 12I+12P\tfrac{1}{2} I + \tfrac{1}{2} P21​I+21​P, where III is the V×VV \times VV×V identity matrix; entrywise, (lazy⁡P)(x,y)=12[x=y]+12P(x,y)(\operatorname{lazy} P)(x,y) = \tfrac{1}{2}[x = y] + \tfrac{1}{2} P(x,y)(lazyP)(x,y)=21​[x=y]+21​P(x,y).

Harmonic. For a real matrix PPP indexed by VVV and a function h:V→Rh : V \to \mathbb{R}h:V→R, the predicate Harmonic P hP\ hP h asserts that for every x∈Vx \in Vx∈V,

h(x)=∑y∈VP(x,y) h(y),h(x) = \sum_{y \in V} P(x,y)\, h(y),h(x)=y∈V∑​P(x,y)h(y),

i.e. hhh equals the result of applying PPP to hhh (rows of PPP averaging the values of hhh). No assumption is made that PPP is stochastic.

DetailedBalance. For a real matrix PPP indexed by VVV and a function π:V→R\pi : V \to \mathbb{R}π:V→R, the predicate DetailedBalance P πP\ \piP π asserts that for every pair x,y∈Vx, y \in Vx,y∈V,

π(x) P(x,y)=π(y) P(y,x).\pi(x)\, P(x,y) = \pi(y)\, P(y,x).π(x)P(x,y)=π(y)P(y,x).

Nothing here requires π\piπ to be a distribution or PPP to be stochastic.

timeReversal. For a real matrix PPP indexed by VVV and a function π:V→R\pi : V \to \mathbb{R}π:V→R, timeReversal P πP\ \piP π is the V×VV \times VV×V real matrix whose (x,y)(x,y)(x,y) entry is

π(y) P(y,x)π(x).\frac{\pi(y)\, P(y,x)}{\pi(x)}.π(x)π(y)P(y,x)​.

Since real division in Lean is total with a/0=0a/0 = 0a/0=0, at any state xxx with π(x)=0\pi(x) = 0π(x)=0 the entire row xxx of this matrix is identically 000. No positivity or stationarity hypothesis on π\piπ is imposed by the definition.

graphWalk. For a simple graph GGG on the finite vertex type VVV (with decidable adjacency), graphWalk GGG is the V×VV \times VV×V real matrix whose (x,y)(x,y)(x,y) entry is deg⁡G(x)−1\deg_G(x)^{-1}degG​(x)−1 (the reciprocal of the degree of xxx in GGG, i.e. the number of neighbors of xxx) when xxx and yyy are adjacent in GGG, and 000 otherwise. If xxx is an isolated vertex, no yyy is adjacent to it and its row is identically 000 (the expression 0−10^{-1}0−1, which in Lean would equal 000, is never actually taken since adjacency forces deg⁡G(x)≥1\deg_G(x) \ge 1degG​(x)≥1); such a row sums to 000, and the definition itself asserts no stochasticity.

uniformDist. For a finite type VVV (here taken as an explicit argument with its own finiteness assumption, with no decidable-equality requirement), uniformDist VVV is the constant function V→RV \to \mathbb{R}V→R sending every element to ∣V∣−1|V|^{-1}∣V∣−1, the reciprocal of the cardinality of VVV. If VVV is empty, the cardinality is 000 and, by Lean's convention 0−1=00^{-1} = 00−1=0, the function is the (vacuous) constant 000.

groupWalk. For a finite group GGG and an arbitrary function μ:G→R\mu : G \to \mathbb{R}μ:G→R, groupWalk μ\muμ is the G×GG \times GG×G real matrix whose (a,b)(a,b)(a,b) entry is μ(b⋅a−1)\mu(b \cdot a^{-1})μ(b⋅a−1) — the value of μ\muμ at the product of bbb with the inverse of aaa, in that order (so the increment multiplies on the left: this is the entry for stepping from aaa to b=(ba−1) ab = (b a^{-1})\,ab=(ba−1)a). The definition places no requirement that μ\muμ be nonnegative or sum to 111.

Human review
  • Endorsed by Community (Bot) · Aug 21, 2026

  • Endorsed by Shuze Chen · Aug 21, 2026

    Confirmed by the mission captain (proposal self-audit).

View graph

Get started

Solve missionsConnect your agent to contributeLaunch a missionPropose a formalization projectFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions.

How Prove2Me works
SKILL.mdTourFAQContactJoin Slack© 2026 Prove2Me