Display (5.9) — J*_β(i) − β J*_β(0) ≤ MK
ProvedArapostathisAC.RossRecurrence.eq_5_9controlled-markov-processdiscounted-costp2o-batch-pfp2bp2o-gran-per-chapterp2o-plan-paperp2o-v1return-time
Consider the countable-state controlled Markov process of §5 with bounded cost, for every and . Let and assume Ross's recurrence condition (5.7): there is with
Then for every and every state ,
This is the upper half of the uniform bound on in Theorem 5.3.
Formalization Note. Under bounded cost, is finite, so its value is used as a real number.
Preamble
import Mathlib import Definitions.Def_ArapostathisAC_RossRecurrence_CMP import Definitions.Def_ArapostathisAC_RossRecurrence_ReturnTime open MeasureTheory ProbabilityTheory open scoped ENNReal
Formal statement
namespace ArapostathisAC.RossRecurrence
/-- Display (5.9) (proof of Theorem 5.3, p. 302): if `c ≤ M₀` on the admissible pairs and the mean
return time to `0` is below `K` under every `f ∈ Π_SD` from every state (5.7), then
`J*_β(i) − β J*_β(0) ≤ M₀ K` for every `β ∈ (0, 1)` and every state `i`. -/
theorem eq_5_9 {A : Type*} [MetricSpace A] [MeasurableSpace A] [BorelSpace A]
(M : CMP A) (M₀ : ℝ) (hc : ∀ i, ∀ a ∈ M.U i, M.c i a ≤ M₀)
(K : ℝ) (hK : 0 < K)
(h57 : ∀ f : StationaryPolicy M, ∀ i, meanReturnTime M f i < ENNReal.ofReal K)
(β : ℝ) (hβ : β ∈ Set.Ioo (0 : ℝ) 1) (i : ℕ) :
(discValue M β i).toReal - β * (discValue M β 0).toReal ≤ M₀ * K := by sorry
end ArapostathisAC.RossRecurrence
Source
Arapostathis, Borkar, Fernández-Gaucherand, Ghosh, Marcus, Discrete-time controlled Markov processes with average cost criterion: a survey, SIAM J. Control Optim. 31(2) (1993), p. 302, proof of Theorem 5.3, (5.9)
Human review
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.