The trajectory law is a finite mixture of product reward laws along state–action paths
ProvedTPG.trajLaw_eq_sum_mapSetting. is a finite state space, actions are , time is (the source's time ). The environment has transition matrices , initial distribution and reward laws on . For a sequence of treatment probabilities, and denote the law of the trajectory when , the actions are independent with , , and the rewards are conditionally independent with (Env.trajLaw T q); the Bernoulli policy is .
Lemma. For every horizon and every real sequence , the trajectory law is the finite mixture
where
is the path weight and inserts a reward vector into the trajectory with state path and action path .
This is the working form of the definition: conditionally on the state and action paths, the rewards are independent with the prescribed laws. Integrals against the trajectory law reduce through it to finite sums of integrals against product measures.
import Mathlib import Definitions.Def_TPG_Model import Definitions.Def_TPG_Markov open MeasureTheory ProbabilityTheory open scoped BigOperators ENNReal
namespace TPG
theorem trajLaw_eq_sum_map {S : Type} [Fintype S] [MeasurableSpace S] [MeasurableSingletonClass S]
(E : Env S) (T : ℕ) (q : ℕ → ℝ) :
E.trajLaw T q
= ∑ p : (Fin T → S) × (Fin T → Bool),
ENNReal.ofReal (E.pathWeight T q p.1 p.2) •
(E.rewardLaw p.1 p.2).map (embedPath p.1 p.2) := by sorry
end TPG