Theorem 5.9 — under Sennott's Assumptions 5.14–5.16 an AC-optimal stationary policy exists
ProvedArapostathisAC.SennottACOI.theorem_5_9Consider a controlled Markov process with countable state space , nonempty compact admissible action sets , a nonnegative one-stage cost and transition probabilities , with and continuous on . Let be the optimal -discounted cost, , and suppose:
- (Assumption 5.14) for every and ;
- (Assumption 5.15) there is a nonnegative integer with for all and ;
- (Assumption 5.16) there is with for all and , and for each an action with .
Then there exists a stationary deterministic policy that is AC-optimal:
the infimum ranging over all admissible policies, history dependent and randomized.
This is Sennott's existence theorem for average-cost optimal stationary policies with unbounded costs, in the form surveyed by Arapostathis et al.; it requires no bounded solution of the average cost optimality equation, only the one-sided conditions above.
Formalization Note The average costs are s in and the optimal average cost is the infimum over the class of all admissible policies. Assumptions 5.14–5.16 are the definitions Assumption5_14, Assumption5_15, Assumption5_16; the differential value is used only together with Assumption 5.14, so it is never read off an infinite . The proof also shows that is a constant and that attains an average cost optimality inequality; these are stated in the milestones, not in this goal.
import Mathlib import Definitions.Def_ArapostathisAC_SennottACOI_CMP import Definitions.Def_ArapostathisAC_SennottACOI_Assumptions open MeasureTheory ProbabilityTheory Filter Topology open scoped ENNReal
namespace ArapostathisAC.SennottACOI
variable {A : Type*} [MetricSpace A] [MeasurableSpace A] [BorelSpace A]
/-- Theorem 5.9 (p. 308): under Assumptions 5.14–5.16 there exists an AC-optimal stationary
deterministic policy `f ∈ Π_SD`, i.e. `J(i, f) = J*(i) = inf_{π ∈ Π} J(i, π)` for every state
`i`, the infimum ranging over all admissible (history-dependent, randomized) policies. -/
theorem theorem_5_9 (M : ArapostathisAC.VanishingDiscount.CMP A) (h14 : Assumption5_14 M) (h15 : Assumption5_15 M)
(h16 : Assumption5_16 M) :
∃ f : StationaryPolicy M, ArapostathisAC.VanishingDiscount.IsAvgOptimal M f := by sorry
end ArapostathisAC.SennottACOI
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.