Proposition C.4.3 — steady state probabilities along an AS: zero limits off positive recurrent states, subsequential limits bπ on a class
ProvedSennottDP.ChainASM.as_steady_state_limitsapproximating-sequencemarkov-chainp2o-batch-b23ap2o-gran-per-chapterp2o-plan-bookp2o-v1steady-state
Let be an approximating sequence for the Markov chain on a denumerable state space , and write for the steady state probability of in (set to for ). Then:
- If is transient or null recurrent, then .
- Let be a positive recurrent class of . Given a sequence , there exist a subsequence of and a constant with such that
The constant measures how much of the steady state mass of the class survives the approximation; for all sequences is equivalent to on .
Formalization Note "Transient or null recurrent" is encoded as "not positive recurrent", i.e. . The sequence is a strictly increasing sequence of indices and the subsequence is for a strictly increasing .
Preamble
import Mathlib import Definitions.Def_SennottDP_ChainASM_MarkovChain import Definitions.Def_SennottDP_ChainASM_ApproxSeq open scoped ENNReal NNReal open Filter Topology open Classical
Formal statement
namespace SennottDP.ChainASM
theorem as_steady_state_limits {S : Type*} [Countable S] [Infinite S]
(Γ : MC S) (AS : ApproxSeq Γ) :
(∀ i : S, ¬ Γ.PosRecurrent i →
Tendsto (fun N => AS.steadyStateN N i) atTop (𝓝 (Γ.steadyState i)) ∧
Γ.steadyState i = 0) ∧
(∀ R : Set S, Γ.IsPosRecClass R → ∀ Nr : ℕ → ℕ, StrictMono Nr →
∃ φ : ℕ → ℕ, StrictMono φ ∧ ∃ b : ℝ≥0∞, b ≤ 1 ∧
∀ i ∈ R, Tendsto (fun s => AS.steadyStateN (Nr (φ s)) i) atTop
(𝓝 (b * Γ.steadyState i))) := by sorry
end SennottDP.ChainASM
Source
Sennott, Stochastic Dynamic Programming and the Control of Queueing Systems (Wiley, 1999), p. 303, Proposition C.4.3
Human review
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.