Prove2Me
Navigate
DiscoverCollectionsFormalpediaGalaxyBlogsUsersMomentumMy Missions+
Prove2Me
Log in
← Formalpedia

Display (5.8) — first-return bound J*_β(i) ≤ M E[τ] + J*_β(0) E[β^τ]

Proved
ArapostathisAC.RossRecurrence.eq_5_8

by mikedeng1 · Oct 5, 2026 · Mathlib 0df444a (Lean v4.33.1)

controlled-markov-processdiscounted-costp2o-batch-pfp2bp2o-gran-per-chapterp2o-plan-paperp2o-v1return-time

Consider the countable-state controlled Markov process of §5. Assume the cost is bounded: there is MMM with c(i,a)≤Mc(i,a)\le Mc(i,a)≤M for every state iii and every a∈U(i)a\in U(i)a∈U(i), as §5.1 assumes. Let β∈(0,1)\beta\in(0,1)β∈(0,1) and let fβ∈ΠSDf_\beta\in\Pi_{SD}fβ​∈ΠSD​ be β\betaβ-discount optimal. Let τ=min⁡{t≥1:Xt=0}\tau=\min\{t\ge1: X_t=0\}τ=min{t≥1:Xt​=0} be the return time to state 000, with βτ=0\beta^\tau=0βτ=0 on {τ=∞}\{\tau=\infty\}{τ=∞}. Then for every state iii,

Jβ∗(i)≤M Eifβ[τ]+Jβ∗(0) Eifβ[βτ].J^*_\beta(i)\le M\,E^{f_\beta}_i[\tau]+J^*_\beta(0)\,E^{f_\beta}_i[\beta^\tau].Jβ∗​(i)≤MEifβ​​[τ]+Jβ∗​(0)Eifβ​​[βτ].

This is the last line of display (5.8) in the proof of Theorem 5.3. It splits the discounted cost of fβf_\betafβ​ at the first return to 000: before τ\tauτ the cost is at most MMM per step, and after τ\tauτ the process restarts from 000 under the optimal policy fβf_\betafβ​.

Formalization Note. The paper's intermediate line writes the first sum as ∑t=0τ−1βnc(Xt,fβ(Xt))\sum_{t=0}^{\tau-1}\beta^n c(X_t,f_\beta(X_t))∑t=0τ−1​βnc(Xt​,fβ​(Xt​)). The exponent should be ttt (a typo), and only the final inequality is stated here. All quantities are in [0,∞][0,\infty][0,∞]; MEifβ[τ]M E^{f_\beta}_i[\tau]MEifβ​​[τ] is MMM times the mean return time, possibly +∞+\infty+∞. The bound on ccc is the standing hypothesis of §5.1.

Preamble
import Mathlib
import Definitions.Def_ArapostathisAC_RossRecurrence_CMP
import Definitions.Def_ArapostathisAC_RossRecurrence_ReturnTime
open MeasureTheory ProbabilityTheory
open scoped ENNReal
Formal statement
namespace ArapostathisAC.RossRecurrence

/-- Display (5.8) (proof of Theorem 5.3, p. 302): if `c ≤ M₀` on the admissible pairs and
`f ∈ Π_SD` is `β`-discount optimal, then `J*_β(i) ≤ M₀ E^f_i[τ] + J*_β(0) E^f_i[β^τ]`. -/
theorem eq_5_8 {A : Type*} [MetricSpace A] [MeasurableSpace A] [BorelSpace A]
    (M : CMP A) (M₀ : ℝ) (hc : ∀ i, ∀ a ∈ M.U i, M.c i a ≤ M₀)
    (β : ℝ) (hβ : β ∈ Set.Ioo (0 : ℝ) 1) (f : StationaryPolicy M) (hf : IsDiscOptimal M f β)
    (i : ℕ) :
    discValue M β i ≤
      ENNReal.ofReal M₀ * meanReturnTime M f i + discValue M β 0 * expDiscAtReturn M f β i := by sorry

end ArapostathisAC.RossRecurrence
Source
Arapostathis, Borkar, Fernández-Gaucherand, Ghosh, Marcus, Discrete-time controlled Markov processes with average cost criterion: a survey, SIAM J. Control Optim. 31(2) (1993), p. 302, proof of Theorem 5.3, (5.8)
Human review
  • Endorsed by Shuze Chen · Oct 5, 2026

    Confirmed by the moderator at approval.

  • Endorsed by mikedeng1 · Oct 5, 2026

    Confirmed by the mission captain (proposal self-audit).

View graph

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paperProve2Me Team
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me