Jensen step — J*_β(i) ≥ J*_β(0) E[β^τ] ≥ J*_β(0) β^K
ProvedArapostathisAC.RossRecurrence.jensen_stepdiscounted-costjensen-inequalityp2o-batch-pfp2bp2o-gran-per-chapterp2o-plan-paperp2o-v1return-time
Consider the countable-state controlled Markov process of §5 with bounded cost, for every and , under Ross's recurrence condition (5.7): for all and , with and . Let and let be -discount optimal. Then for every state ,
This is the unnumbered display after (5.9) in the proof of Theorem 5.3. It gives the lower half of the uniform bound on .
Formalization Note. on , and is a real power since is real. The two inequalities are stated as a conjunction in . The bounded-cost hypothesis is the standing assumption of §5.1.
Preamble
import Mathlib import Definitions.Def_ArapostathisAC_RossRecurrence_CMP import Definitions.Def_ArapostathisAC_RossRecurrence_ReturnTime open MeasureTheory ProbabilityTheory open scoped ENNReal
Formal statement
namespace ArapostathisAC.RossRecurrence
/-- The Jensen step (proof of Theorem 5.3, p. 302, display after (5.9)): under bounded costs and
(5.7), for a `β`-discount optimal `f ∈ Π_SD`,
`J*_β(i) ≥ J*_β(0) E^f_i[β^τ] ≥ J*_β(0) β^K`. -/
theorem jensen_step {A : Type*} [MetricSpace A] [MeasurableSpace A] [BorelSpace A]
(M : CMP A) (M₀ : ℝ) (hc : ∀ i, ∀ a ∈ M.U i, M.c i a ≤ M₀)
(K : ℝ) (hK : 0 < K)
(h57 : ∀ f : StationaryPolicy M, ∀ i, meanReturnTime M f i < ENNReal.ofReal K)
(β : ℝ) (hβ : β ∈ Set.Ioo (0 : ℝ) 1) (f : StationaryPolicy M) (hf : IsDiscOptimal M f β)
(i : ℕ) :
discValue M β 0 * expDiscAtReturn M f β i ≤ discValue M β i ∧
discValue M β 0 * ENNReal.ofReal (β ^ K) ≤ discValue M β 0 * expDiscAtReturn M f β i := by sorry
end ArapostathisAC.RossRecurrence
Source
Arapostathis, Borkar, Fernández-Gaucherand, Ghosh, Marcus, Discrete-time controlled Markov processes with average cost criterion: a survey, SIAM J. Control Optim. 31(2) (1993), p. 302, proof of Theorem 5.3, display after (5.9)
Human review
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.