Finite Markov chains: transition matrices, stationarity, irreducibility, period, reversibility
Definitionmm_basicThis file sets up the basic vocabulary of finite Markov chains, following Chapters 1-2 of Levin-Peres-Wilmer. A Markov chain on a finite state space is presented by its transition matrix: a matrix with nonnegative entries whose rows sum to one. A probability distribution on is a nonnegative vector of total mass one, acting on the right of as a row vector, and a distribution is stationary when it is fixed by one step of the chain, .
The chain is irreducible if for every pair of states there is a with .
The period of a state is the greatest common divisor of its return-time set , formalized as the largest natural number dividing every element of (a state with empty return set receives the junk value ).
The chain is aperiodic when every state has period one. The file further introduces: the lazy version of a chain;
harmonic functions, those with at every state;
the detailed balance equations , whose validity makes the chain reversible;
the time reversal , whose rows vanish where since division is total;
simple random walk on a graph, which from moves to a uniformly chosen neighbour of ; the uniform distribution on ;
and the random walk on a finite group with increment distribution , which steps from to with probability -- equivalently, its transition probability from to is .
import Mathlib.Data.Matrix.Basic
import Mathlib.Data.Real.Basic
import Mathlib.Data.Nat.Lattice
import Mathlib.Combinatorics.SimpleGraph.DegreeSum
/-!
Basic vocabulary of finite Markov chains, following Levin–Peres–Wilmer,
*Markov Chains and Mixing Times*, Chapter 1.
A chain on a finite state space `V` is presented by its transition matrix
`P : Matrix V V ℝ`, acting on the right of row vectors: a distribution `μ`
evolves to `μ ᵥ* P` in one step, and the `t`-step transition probabilities
are the entries of `P ^ t`.
-/
namespace MarkovMixing
noncomputable section
open scoped BigOperators
variable {V : Type*} [Fintype V] [DecidableEq V]
/-- A matrix is **stochastic** (a transition matrix) if its entries are
nonnegative and every row sums to `1` (LPW §1.1, Eq. (1.1)). -/
def IsStochastic (P : Matrix V V ℝ) : Prop :=
(∀ x y, 0 ≤ P x y) ∧ ∀ x, ∑ y, P x y = 1
/-- A **probability distribution** on the finite state space `V`, presented as a
row vector: nonnegative entries summing to `1`. -/
def IsDist (μ : V → ℝ) : Prop :=
(∀ x, 0 ≤ μ x) ∧ ∑ x, μ x = 1
/-- A distribution `π` is **stationary** for `P` if `π = π P` (LPW §1.1, Eq. (1.4)). -/
def IsStationary (P : Matrix V V ℝ) (π : V → ℝ) : Prop :=
IsDist π ∧ Matrix.vecMul π P = π
/-- A chain is **irreducible** if any state can reach any other: for all
`x y` there is a `t` with `P^t(x,y) > 0` (LPW §1.3). -/
def Irreducible (P : Matrix V V ℝ) : Prop :=
∀ x y : V, ∃ t : ℕ, 0 < (P ^ t) x y
/-- `T(x) = {t ≥ 1 : P^t(x,x) > 0}`, the set of possible return times to `x`
(LPW §1.3). -/
def returnSet (P : Matrix V V ℝ) (x : V) : Set ℕ :=
{t : ℕ | 1 ≤ t ∧ 0 < (P ^ t) x x}
/-- The **period** of a state `x` is `gcd T(x)` (LPW §1.3). It is formalized as
the largest natural number dividing every element of `returnSet P x` (which is
the gcd whenever `T(x) ≠ ∅`, and junk-value `0` when `T(x) = ∅`). -/
def period (P : Matrix V V ℝ) (x : V) : ℕ :=
sSup {d : ℕ | ∀ t ∈ returnSet P x, d ∣ t}
/-- A chain is **aperiodic** if every state has period `1` (LPW §1.3). -/
def Aperiodic (P : Matrix V V ℝ) : Prop :=
∀ x : V, period P x = 1
/-- The **lazy version** `Q = (I + P)/2` of a chain (LPW §1.3). -/
def lazy (P : Matrix V V ℝ) : Matrix V V ℝ :=
(2⁻¹ : ℝ) • (1 : Matrix V V ℝ) + (2⁻¹ : ℝ) • P
/-- A function `h` is **harmonic** for `P` (at every state) if
`h(x) = ∑_y P(x,y) h(y)` (LPW §1.5.4, Eq. (1.28)). -/
def Harmonic (P : Matrix V V ℝ) (h : V → ℝ) : Prop :=
∀ x : V, h x = ∑ y, P x y * h y
/-- `π` and `P` are in **detailed balance** if `π(x) P(x,y) = π(y) P(y,x)` for
all `x, y`; a chain admitting such a distribution is called reversible
(LPW §1.6, Eq. (1.30)). -/
def DetailedBalance (P : Matrix V V ℝ) (π : V → ℝ) : Prop :=
∀ x y : V, π x * P x y = π y * P y x
/-- The **time reversal** `P̂(x,y) = π(y) P(y,x) / π(x)` of a chain with
stationary distribution `π` (LPW §1.6, Eq. (1.33)). -/
def timeReversal (P : Matrix V V ℝ) (π : V → ℝ) : Matrix V V ℝ :=
fun x y => π y * P y x / π x
/-- **Simple random walk** on a graph `G`: from `x`, move to a uniformly chosen
neighbor of `x` (LPW §1.4, Eq. (1.13)). -/
def graphWalk (G : SimpleGraph V) [DecidableRel G.Adj] : Matrix V V ℝ :=
fun x y => if G.Adj x y then ((G.degree x : ℝ))⁻¹ else 0
/-- The **uniform distribution** on a finite state space. -/
def uniformDist (V : Type*) [Fintype V] : V → ℝ :=
fun _ => (Fintype.card V : ℝ)⁻¹
/-- The **random walk on a finite group** `G` with increment distribution `μ`:
from `a`, move to `h * a` where `h ∼ μ`, so the transition probability from
`a` to `b` is `μ (b * a⁻¹)` (LPW §2.6). -/
def groupWalk {G : Type*} [Group G] [Fintype G] (μ : G → ℝ) : Matrix G G ℝ :=
fun a b => μ (b * a⁻¹)
end
end MarkovMixing
Read-back
What the Lean code literally says, in plain math · claude-fable-5
Read-back: Def_mm_basic (namespace MarkovMixing)
Throughout, is an arbitrary type assumed finite (with decidable equality), and all matrices are matrices with real entries, viewed as functions of two arguments: is written below. Sums range over all elements of the finite type .
IsStochastic. For a real matrix indexed by , the predicate IsStochastic asserts the conjunction of two conditions: every entry is nonnegative, ; and every row sums to one, . (Rows, not columns: the sum for fixed first index runs over the second index .) If is empty, both conditions hold vacuously.
IsDist. For a function , the predicate IsDist asserts that for every and that . If is empty the sum is , so no on an empty type satisfies this.
IsStationary. For a real matrix indexed by and a function , the predicate IsStationary asserts that satisfies IsDist (entrywise nonnegative with total mass , as above) and that the row vector multiplied on the right by equals ; componentwise, for every . No stochasticity assumption on is part of this definition.
Irreducible. For a real matrix indexed by , the predicate Irreducible asserts: for every pair of states there exists a natural number such that the entry of the matrix power is strictly positive, . The exponent is allowed; since is the identity matrix, the case is always witnessed by regardless of , so the definition only imposes a genuine condition on pairs .
returnSet. For a real matrix indexed by and a state , returnSet is the set of natural numbers
i.e. the set of strictly positive times at which the diagonal entry of at is strictly positive. Time is explicitly excluded.
period. For a real matrix indexed by and a state , period is defined as the supremum, taken in the natural numbers, of the set of common divisors of the return set:
where is the returnSet just described. This supremum is Lean's supremum on , which returns for a set that is empty or unbounded. Consequently: if is nonempty, the common divisors form a bounded set and the value is the greatest common divisor of all return times; but if is empty (the state never returns), then every natural number vacuously divides every element, the divisor set is all of , which is unbounded, and the defined period is , not .
Aperiodic. For a real matrix indexed by , the predicate Aperiodic asserts that for every state , with period as defined above (in particular, a state with empty return set has period and so witnesses failure of this predicate).
lazy. For a real matrix indexed by , lazy is the matrix , where is the identity matrix; entrywise, .
Harmonic. For a real matrix indexed by and a function , the predicate Harmonic asserts that for every ,
i.e. equals the result of applying to (rows of averaging the values of ). No assumption is made that is stochastic.
DetailedBalance. For a real matrix indexed by and a function , the predicate DetailedBalance asserts that for every pair ,
Nothing here requires to be a distribution or to be stochastic.
timeReversal. For a real matrix indexed by and a function , timeReversal is the real matrix whose entry is
Since real division in Lean is total with , at any state with the entire row of this matrix is identically . No positivity or stationarity hypothesis on is imposed by the definition.
graphWalk. For a simple graph on the finite vertex type (with decidable adjacency), graphWalk is the real matrix whose entry is (the reciprocal of the degree of in , i.e. the number of neighbors of ) when and are adjacent in , and otherwise. If is an isolated vertex, no is adjacent to it and its row is identically (the expression , which in Lean would equal , is never actually taken since adjacency forces ); such a row sums to , and the definition itself asserts no stochasticity.
uniformDist. For a finite type (here taken as an explicit argument with its own finiteness assumption, with no decidable-equality requirement), uniformDist is the constant function sending every element to , the reciprocal of the cardinality of . If is empty, the cardinality is and, by Lean's convention , the function is the (vacuous) constant .
groupWalk. For a finite group and an arbitrary function , groupWalk is the real matrix whose entry is — the value of at the product of with the inverse of , in that order (so the increment multiplies on the left: this is the entry for stepping from to ). The definition places no requirement that be nonnegative or sum to .
Confirmed by the mission captain (proposal self-audit).