The GATE in matrix form:
ProvedTPG.gate_eq_meanRewardcausal-inferencemarkov-chainnonstationaritypolicy-gradienttreatment-effect
Setting. is a finite state space, actions are , time is (the source's time ). The environment has transition matrices , initial distribution and reward laws on . For a sequence of treatment probabilities, and denote the law of the trajectory when , the actions are independent with , , and the rewards are conditionally independent with (Env.trajLaw T q); the Bernoulli policy is .
Lemma. If for all , then the global average treatment effect satisfies
where and are the constant sequences, so that and .
This is the estimand in the form compared with the expectation of the estimators in the bias bounds.
Preamble
import Mathlib import Definitions.Def_TPG_Model import Definitions.Def_TPG_Estimators import Definitions.Def_TPG_Markov import Definitions.Def_TPG_Bridge open MeasureTheory ProbabilityTheory open scoped BigOperators ENNReal
Formal statement
namespace TPG
theorem gate_eq_meanReward {S : Type} [Fintype S] [DecidableEq S] [MeasurableSpace S] [MeasurableSingletonClass S]
(E : Env S) (T : ℕ) (M : ℝ) (hM : E.RewardBound M) :
E.gate T
= (1 / (T : ℝ)) * ∑ t : Fin T,
(E.meanReward (fun _ => 1) (t : ℕ) - E.meanReward (fun _ => 0) (t : ℕ)) := by sorry
end TPGSource
Johari, Peng, Xing, Estimation of Treatment Effects Under Nonstationarity via the Truncated Policy Gradient Estimator, revised manuscript of October 2026 (revision of arXiv:2506.05308v3, https://arxiv.org/abs/2506.05308); numbering follows the revised manuscript. Auxiliary statement, not a numbered result of the paper: it is the identity E_{L_1^t}[Y_t] = ρ^⊤ P_1^{1→t} r_1 of eq. (30), p. 39 (Appendix F.1), averaged over the horizon.