Lemma 1.2, part 2 — T(π, a, k) is MLR-monotone in π iff πP^a is
ProvedLovejoyPOMDP.Monotone.lemma_1_2_2bayesian-updatelikelihood-ratio-orderp2o-batch-pfp2ap2o-gran-per-chapterp2o-plan-paperp2o-v1pomdp
Consider a finite POMDP as in §2 of the paper, with for all . For every action and observation , the following are equivalent:
- for all in ;
- for all in .
The Bayes update preserves the MLR order of priors exactly when the prediction step does. Together with Lemma 1.3(1) this gives the monotonicity of the posterior in the prior used in Proposition 1.
Formalization Note The equivalence relies on the standing assumption (a field of the model), which makes on .
Preamble
import Mathlib import Definitions.Def_LovejoyPOMDP_Monotone_Model
Formal statement
namespace LovejoyPOMDP.Monotone
/-- Lovejoy, *Some Monotonicity Results for Partially Observed Markov Decision Processes*, Oper. Res. 35(5):736–743 (1987), DOI 10.1287/opre.35.5.736, pp. 738–739, Lemma 1.2, part 2.
For any `a ∈ A` and `k ∈ O`: `T(π, a, k) ≥r T(π', a, k)` for all `π ≥r π'` in `Π(S)` if and only
if `πP^a ≥r π'P^a` for all `π ≥r π'` in `Π(S)`.
**Formalization Note.** The equivalence uses the standing assumption `r^a_{jk} > 0`, which is a
field of `POMDP`. -/
theorem lemma_1_2_2 {S O A : Type*} [Fintype S] [Fintype O] [Fintype A]
[LinearOrder S] [LinearOrder O] [LinearOrder A] [Nonempty S] [Nonempty O] [Nonempty A]
(M : POMDP S O A) (a : A) (k : O) :
(∀ π π' : S → ℝ, π ∈ stdSimplex ℝ S → π' ∈ stdSimplex ℝ S → MLRGE π π' →
MLRGE (M.bayes π a k) (M.bayes π' a k)) ↔
(∀ π π' : S → ℝ, π ∈ stdSimplex ℝ S → π' ∈ stdSimplex ℝ S → MLRGE π π' →
MLRGE (M.predict π a) (M.predict π' a)) := by sorry
end LovejoyPOMDP.Monotone
Source
Lovejoy, Some Monotonicity Results for Partially Observed Markov Decision Processes, Oper. Res. 35(5):736–743 (1987), DOI 10.1287/opre.35.5.736, pp. 738–739, Lemma 1.2, part 2
Human review
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.