Restricting to is the same as fixing the action at time
ProvedTPG.integral_indicator_Z_mulSetting. is a finite state space, actions are , time is (the source's time ). The environment has transition matrices , initial distribution and reward laws on . For a sequence of treatment probabilities, and denote the law of the trajectory when , the actions are independent with , , and the rewards are conditionally independent with (Env.trajLaw T q); the Bernoulli policy is .
Lemma. Let for all , let and , and let be a measurable function of the trajectory that is integrable along every state–action path (as in TPG.integral_trajLaw). Write for the sequence with replaced by (so the action at time is with probability one). Then
This is the mechanism behind every conditional expectation given one action in the source: conditioning on changes one coordinate of the randomization sequence and nothing else.
import Mathlib import Definitions.Def_TPG_Model import Definitions.Def_TPG_Markov open MeasureTheory ProbabilityTheory open scoped BigOperators ENNReal
namespace TPG
theorem integral_indicator_Z_mul {S : Type} [Fintype S] [MeasurableSpace S] [MeasurableSingletonClass S]
(E : Env S) (T : ℕ) (q : ℕ → ℝ) (hq : ∀ t : ℕ, 0 ≤ q t ∧ q t ≤ 1)
(u : Fin T) (z : Bool) (F : Traj S T → ℝ) (hF : Measurable F)
(hint : ∀ p : (Fin T → S) × (Fin T → Bool),
Integrable (fun y : Fin T → ℝ => F (embedPath p.1 p.2 y)) (E.rewardLaw p.1 p.2)) :
∫ ω, (if Z u ω = z then F ω else 0) ∂(E.trajLaw T q)
= (if z then q (u : ℕ) else 1 - q (u : ℕ))
* ∫ ω, F ω ∂(E.trajLaw T (Function.update q (u : ℕ) (if z then 1 else 0))) := by sorry
end TPG