Theorem 9.3.7 — stability of the value and of the optimal policies under utility perturbation
ProvedMDPFinance.JumpMarkets.theorem_9_3_7Theorem 9.3.7 (p. 290).
a) If and are two utility functions with corresponding value functions and , then . b) Let be a sequence of utility functions with . Then it holds for all , i.e. in particular, the limit of a sequence with for all defines an optimal stationary policy for the given model (with utility function ).
The practical payoff of the contracting structure: a utility function is never known exactly, and this says the answer degrades gracefully. a) is a Lipschitz estimate for the value with the explicit constant ; b) is the corresponding statement for the optimal policies, which is strictly stronger and does not follow from a) — value functions can be close while argmax sets are far apart.
b) is an upper-limit statement, not a limit. is the book's own
definition (p. 201): an accumulation point of some sequence with
— a statement about a sequence of points, not a Filter.limsup of sets, and
rendered here as chunk 07a renders it. Both halves are the content:
says approximating policies have accumulation points, and the inclusion says every such point is
optimal for the true utility.
a) is stated in its -form against the bounding function rather than by forming , which is itself a supremum.
Moderation note. In the draft the approximating markets M_n were arbitrary (not the same market with a different utility — refutable), their V_n arbitrary fixed points, and Ls was in an arbitrary topology. Now M_n share the market data, V, Ṽ, V^{(n)} are the value functions, a) is a two-sided [0,∞] bound with the constant e^{T\barμ}/(1 − α_b), and b) is ∅ ≠ Ls A^*_n(t,x) ⊂ A^*(t,x) over the relaxed controls with the Young topology.
import Mathlib import Definitions.Def_MDPFinance_JumpMarkets_JumpMarket open MeasureTheory Filter Topology open scoped ENNReal
namespace MDPFinance.JumpMarkets
/-- Two jump markets with the same market data, differing only in the utility. -/
def JumpMarket.SameMarket {d : ℕ} (M M' : JumpMarket d) : Prop :=
M'.rho = M.rho ∧ M'.mu = M.mu ∧ M'.lam = M.lam ∧ M'.QY = M.QY ∧ M'.T = M.T
/-- **Theorem 9.3.7** (p. 290), for the contracting model (`b = b_γ`, `α_b < 1`). a) If `U`,
`Ũ` are two utility functions (same market) with value functions `V`, `Ṽ`, then `‖V - Ṽ‖_b ≤
‖U - Ũ‖_b e^{T\barμ}/(1-α_b)` (in `≤`-form against `b`, two-sided in `[0,∞]`). b) If `(U^{(n)})`
are utility functions with `‖U^{(n)} - U‖_b → 0` and `V^{(n)}` their value functions, then
`∅ ≠ Ls A^*_n(t,x) ⊂ A^*(t,x)` on `E`, the maximum-point sets over the relaxed controls `R`
with the Young topology. -/
theorem theorem_9_3_7 {d : ℕ} (M Mtilde : JumpMarket d)
(gamma alpha : ℝ) (hb : M.IsBoundingFunction (M.bfun gamma) alpha)
(hbt : Mtilde.IsBoundingFunction (M.bfun gamma) alpha) (halpha : alpha < 1)
(hsame : M.SameMarket Mtilde)
(Pr : HistPolicy d → ℝ × ℝ → Measure (ℕ → ℝ × ℝ))
(hPr : ∀ g, M.IsChainLaw g (Pr g)) :
(∀ C : ℝ, (∀ p ∈ M.E, |M.U p.2 - Mtilde.U p.2| ≤ C * M.bfun gamma p) →
∀ p ∈ M.E,
M.V Pr p ≤ Mtilde.V Pr p +
ENNReal.ofReal (C * Real.exp (M.T * M.mubar) / (1 - alpha) * M.bfun gamma p) ∧
Mtilde.V Pr p ≤ M.V Pr p +
ENNReal.ofReal (C * Real.exp (M.T * M.mubar) / (1 - alpha) * M.bfun gamma p)) ∧
(∀ Mn : ℕ → JumpMarket d, (∀ n, M.SameMarket (Mn n)) →
(∀ n, (Mn n).IsBoundingFunction (M.bfun gamma) alpha) →
(∀ ε > (0 : ℝ), ∀ᶠ n in atTop, ∀ p ∈ M.E,
|(Mn n).U p.2 - M.U p.2| ≤ ε * M.bfun gamma p) →
∀ p ∈ M.E,
(LsSeq (fun n => (Mn n).AstarRel ((Mn n).V Pr) p)).Nonempty ∧
LsSeq (fun n => (Mn n).AstarRel ((Mn n).V Pr) p) ⊆ M.AstarRel (M.V Pr) p) := by sorry
end MDPFinance.JumpMarkets
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.