Corollary 2(a) — the robust value of a stationary policy (d, d, …) is the unique bounded solution of (29)
ProvedRobustDP.Discounted.corollary2a_policy_valuedynamic-programmingp2o-batch-pfp2ap2o-gran-per-chapterp2o-plan-paperp2o-v1policy-evaluationrobust-optimization
Let be a discounted ambiguous MDP, let be a deterministic Markov decision rule, and let . Then is bounded, it solves
and every bounded solution of (29) equals .
This is robust policy evaluation: it is the base of Lemmas 2 and 3 and of the robust policy iteration algorithm.
Formalization Note "Deterministic decision rule" is a deterministic Markov rule , as throughout Section 3. Uniqueness is among bounded functions. is the robust value against the dynamic adversary.
Preamble
import Mathlib import Definitions.Def_RobustDP_Discounted_Model import Definitions.Def_RobustDP_Discounted_Value import Definitions.Def_RobustDP_Discounted_Bellman
Formal statement
namespace RobustDP.Discounted
theorem corollary2a_policy_value {S A : Type*} [Countable S] [Countable A] (M : Model S A)
(d : DecisionRule M) :
IsBounded (V M (stationary M d)) ∧
(∀ s, V M (stationary M d) s = Q M (V M (stationary M d)) s (d.1 s)) ∧
∀ U : S → ℝ, IsBounded U → (∀ s, U s = Q M U s (d.1 s)) → U = V M (stationary M d) := by sorry
end RobustDP.Discounted
Source
Iyengar, Robust dynamic programming, CORC Tech Report TR-2002-07 (rev. May 4, 2004), p. 11, Corollary 2(a), eq. (29)
Human review
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.