Proposition 9.5 — under (P), (N), (D)
OpenBertsekasShreve.BorelInfinite.Jbarstar_eq_expect_Jstarborel-modeldynamic-programming
Let (SM) be an infinite horizon stochastic optimal control model on nonempty Borel spaces (Definitions 8.1, 9.1), with optimal cost over all universally measurable policies, and let (DM) be the associated deterministic model on (Definition 9.4) with optimal cost and operator . Assume one of (P) on , (N) on , (D) and on .
Then for every ,
This is one of the two key relationships between (SM) and (DM): the optimal cost of the deterministic model is the average of the stochastic optimal cost.
Formalization Note The integral is the extended integral with (DupacovaWets.Consistency.expect).
Preamble
import Mathlib import Definitions.Def_BertsekasShreve_BorelInfinite_Analytic import Definitions.Def_BertsekasShreve_BorelInfinite_ExtArith import Definitions.Def_BertsekasShreve_BorelInfinite_Model import Definitions.Def_BertsekasShreve_BorelInfinite_Policy import Definitions.Def_BertsekasShreve_BorelInfinite_DetModel open MeasureTheory ProbabilityTheory Filter Topology
Formal statement
namespace BertsekasShreve.BorelInfinite
/-- **Proposition 9.5** (p. 223). (P)(N)(D) For every `p ∈ P(S)`, `J̄*(p) = ∫ J*(x) p(dx)`. -/
theorem Jbarstar_eq_expect_Jstar {S C W : Type*}
[TopologicalSpace S] [BertsekasShreve.AnalyticSelection.IsBorelSpace S] [MeasurableSpace S] [BorelSpace S] [Nonempty S]
[TopologicalSpace C] [BertsekasShreve.AnalyticSelection.IsBorelSpace C] [MeasurableSpace C] [BorelSpace C] [Nonempty C]
[TopologicalSpace W] [BertsekasShreve.AnalyticSelection.IsBorelSpace W] [MeasurableSpace W] [BorelSpace W] [Nonempty W]
(M : Model S C W)
(hcase : M.CaseP ∨ M.CaseN ∨ M.CaseD) (p : Measure S) [IsProbabilityMeasure p] :
M.Jbarstar p = DupacovaWets.Consistency.expect p M.Jstar := by sorry
end BertsekasShreve.BorelInfinite
Source
Bertsekas & Shreve, Stochastic Optimal Control: The Discrete-Time Case, Athena Scientific 1996, p. 223, Proposition 9.5 of Chapter 9