Lemma 1.2, part 3 — sufficient conditions for T(π, a, k) to be MLR-nondecreasing in the action
ProvedLovejoyPOMDP.Monotone.lemma_1_2_3bayesian-updatelikelihood-ratio-orderp2o-batch-pfp2ap2o-gran-per-chapterp2o-plan-paperp2o-v1pomdp
Consider a finite POMDP as in §2 of the paper, with a completely ordered action set . Fix a belief and an observation . Suppose that for all in :
- , and
- for all in .
Then
After observing , the posterior obtained under a larger action is larger in the likelihood ratio order. This is the step of Proposition 2's proof that compares continuation values across actions.
Preamble
import Mathlib import Definitions.Def_LovejoyPOMDP_Monotone_Model
Formal statement
namespace LovejoyPOMDP.Monotone
/-- Lovejoy, *Some Monotonicity Results for Partially Observed Markov Decision Processes*, Oper. Res. 35(5):736–743 (1987), DOI 10.1287/opre.35.5.736, p. 739, Lemma 1.2, part 3.
For any `π ∈ Π(S)` and `k ∈ O`: `T(π, a, k) ≥r T(π, a', k)` for all `a ≥ a'` in `A` if for all such
`a` and `a'`: (a) `πP^a ≥r πP^{a'}`; (b) `r^a_{jk} r^{a'}_{j'k} ≥ r^{a'}_{jk} r^a_{j'k}` for all
`j ≥ j'` in `S`. -/
theorem lemma_1_2_3 {S O A : Type*} [Fintype S] [Fintype O] [Fintype A]
[LinearOrder S] [LinearOrder O] [LinearOrder A] [Nonempty S] [Nonempty O] [Nonempty A]
(M : POMDP S O A)
{π : S → ℝ} (hπ : π ∈ stdSimplex ℝ S) (k : O)
(ha : ∀ a a' : A, a' ≤ a → MLRGE (M.predict π a) (M.predict π a'))
(hb : ∀ a a' : A, a' ≤ a → ∀ j j' : S, j' ≤ j →
M.R a' j k * M.R a j' k ≤ M.R a j k * M.R a' j' k) :
∀ a a' : A, a' ≤ a → MLRGE (M.bayes π a k) (M.bayes π a' k) := by sorry
end LovejoyPOMDP.Monotone
Source
Lovejoy, Some Monotonicity Results for Partially Observed Markov Decision Processes, Oper. Res. 35(5):736–743 (1987), DOI 10.1287/opre.35.5.736, p. 739, Lemma 1.2, part 3
Human review
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.