Lemma 4 — consistency of the ideal HAC estimator
ProvedTPG.lemma_4_ideal_hac_consistencySetting. is a finite state space and the action is control or treatment. Time is , which is the source's time . Under action at time the state moves according to a transition matrix , i.e. ; the matrices may depend on arbitrarily. The initial state has distribution , and the reward has distribution on , with for all and conditional independence from everything else given . Under the policy every action is treatment with probability , independently of everything else; , and refer to the law of the trajectory under .
Asymptotic setting. The truncation size is fixed and the horizon ; for each the trajectory is generated under the experimental policy . Assumption 1 holds for all times with one constant . Write and
Assumption 2 (asymptotic variance): as , with .
The bandwidth is a deterministic integer sequence with for every , and , and .
The ideal HAC estimator centers at the time-specific means: with ,
Lemma. Under Assumptions 1 and 2, as .
The ideal estimator is not computable from data, since the means are unknown; the lemma is the first step of the consistency proof for the feasible estimator of Theorem 2.
Formalization Note Convergence in probability is stated as for every , with the probability taken under the horizon- law. The limit is a parameter constrained by Assumption 2.
import Mathlib import Definitions.Def_TPG_Model import Definitions.Def_TPG_Estimators import Definitions.Def_TPG_Inference open MeasureTheory ProbabilityTheory open scoped BigOperators ENNReal
namespace TPG
theorem lemma_4_ideal_hac_consistency {S : Type} [Fintype S] [MeasurableSpace S] [MeasurableSingletonClass S]
(E : Env S) (k : ℕ) (γ M v : ℝ) (L : ℕ → ℕ)
(hγ0 : 0 < γ) (hγ1 : γ < 1) (hmix : ∀ T : ℕ, E.Mixing T γ) (hM : E.RewardBound M)
(hvar : E.AsymptoticVariance k v) (hL : IsBandwidth L) :
E.TendstoInProb (fun (T : ℕ) (ω : Traj S T) => E.hacIdeal T k (L T) ω) v := by sorry
end TPGRead-back
What the Lean code literally says, in plain math · claude-opus-5-5
Read-back of the theorem TPG.lemma_4_ideal_hac_consistency.
1. Data quantified over. The statement is asserted for every choice of the following objects.
-
A finite set , carrying a -algebra in which every singleton is measurable. (As is finite, this forces every subset of to be measurable. is not assumed to be non-empty.)
-
An environment on , which is a bundle of three things:
- for every time and every (a Boolean; = true, = false), a real matrix all of whose entries are and each of whose rows sums to ;
- a function with for every and ;
- for every and , a Borel probability measure on .
The matrices are indexed by all and do not depend on any horizon. If is empty, no environment exists (the requirement cannot hold), so the statement is vacuous for empty .
-
A natural number (the value is allowed), three real numbers , , , and a function .
2. The probability spaces (a different one for each ). For each let
be the set of trajectories of length , with the product -algebra (the given -algebra on , all subsets of , the Borel -algebra on ). For and write for the three coordinates at time .
The measure on is the finite mixture
with weights
Here denotes a point mass, is the entry of the matrix in row and column , and in the last product the factor attached to time is if and if (the underlying construction allows a per-time probability in place of ; in this theorem it is the constant ). With the constant every weight is non-negative, so the clipping changes nothing, and the weights sum to , so is a probability measure.
In words, under : are independent fair coin flips; has distribution ; given the coin flips, is a time-inhomogeneous Markov chain in which the step from time to time uses the transition matrix (rows are the current state, columns the next state); and, conditionally on the whole path , the real numbers are independent with distributed as .
3. The statistics. Fix and . Put and (in the code: ). For define
and its mean under the length- law,
For a lag define the mean-centred lag- product average
and, for a truncation level ,
The random variable appearing in the conclusion is , i.e. the truncation level is . Note that the centring uses the exact means (numbers determined by , , ), not sample averages.
4. Hypotheses. All of the following are assumed.
-
and .
-
(Contraction of every one-step kernel.) For every , every with , every , and every pair of functions that are probability vectors (all values , values summing to ),
Since ranges over all natural numbers, this is the same as requiring the inequality for every (the instance is empty).
-
(Reward bound.) For every and : .
-
(Asymptotic variance.) , and
where for each the variance is that of the function on under .
- (Conditions on .) All three of: (a) for every with , ; (b) as ; (c) as , meaning there are a real constant and a such that for all (with the real power).
5. Conclusion. For every real ,
the limit being taken in . That is, "converges in probability to " in the sense that the -th term is evaluated on the -th probability space ; there is no single probability space carrying all at once.
6. Conventions and boundary cases contained in the literal statement.
- The sample space and the measure change with the index of the limit, both in hypothesis 4 and in the conclusion. The coin-flip probability is fixed at throughout, and the weights are fixed at .
- The event in the conclusion uses the non-strict inequality .
- is a fixed natural number (it does not vary with ); no relation between and or is assumed. For , . For the window is cut off at time .
- Each is normalised by , not by the number of terms; for the sum is empty and . By hypothesis 5(a) the lags used satisfy when . If (forced for ) the lag sum is empty and .
- : is a single point with unit mass, is read as , all sums over are empty, so every statistic equals ; is unconstrained; the real power is . None of this affects limits as .
- The integral defining is a Bochner integral, which by convention is if the integrand is not integrable. The variance in hypothesis 4 is computed as a -valued integral and then converted to a real number with . Under hypothesis 3 each lies in -almost surely, so almost surely.
- No sign condition is imposed on . If the interval is empty and hypothesis 3 cannot hold. If then every almost surely, the variance in hypothesis 4 is for all , and hypothesis 4 () cannot hold. So the statement has content only for .
- The existence of the limit and its strict positivity are assumptions (hypothesis 4), not conclusions.
- Hypothesis 2 is imposed separately on each single matrix , for both values of and every time , with one common constant .
- This theorem has no hypothesis on how the means vary with .
Confirmed by the mission captain (proposal self-audit).