Prove2Me
Navigate
DiscoverCollectionsFormalpediaGalaxyBlogsUsersMomentumMy Missions+
Prove2Me
Log in
← Formalpedia

Posterior diagonal representation for the Thompson-sampling information ratio

Proved
BanditAlgorithm.bayesian_ts_conditional_diagonal_representation

by Harry_Xu · Aug 2, 2026 · Mathlib c5ea003 (Lean v4.30.0)

banditsconditional-probabilityinformation-ratiothompson-sampling

Consider Bayesian Thompson sampling with kkk actions and rewards in [0,1][0,1][0,1]. Fix a round ttt and condition on the history Ht=hH_t=hHt​=h. Let ph(a)=P(A∗=a∣Ht=h)p_h(a)=\mathbb P(A^*=a\mid H_t=h)ph​(a)=P(A∗=a∣Ht​=h). For almost every history, there are conditional reward laws Ph,aP_{h,a}Ph,a​ (given A∗=aA^*=aA∗=a) and reference reward laws Mh,aM_{h,a}Mh,a​ such that

rt(h)=∑aph(a)(EPh,a[Y]−EMh,a[Y])r_t(h)=\sum_a p_h(a)\left(\mathbb E_{P_{h,a}}[Y]-\mathbb E_{M_{h,a}}[Y]\right)rt​(h)=a∑​ph​(a)(EPh,a​​[Y]−EMh,a​​[Y])

and

∑aph(a)2D(Ph,a∥Mh,a)≤I(A∗;(At,Xt,At)∣Ht=h).\sum_a p_h(a)^2D(P_{h,a}\|M_{h,a})\le I(A^*;(A_t,X_{t,A_t})\mid H_t=h).a∑​ph​(a)2D(Ph,a​∥Mh,a​)≤I(A∗;(At​,Xt,At​​)∣Ht​=h).

All displayed laws are probability measures and the divergences are finite. This is the posterior diagonal decomposition used in the information-ratio proof for Thompson sampling.

Formalization Note The reward laws are measures on the subtype [0,1][0,1][0,1], which makes the bounded-reward hypotheses explicit.

Preamble
import Definitions.Def_BayesianTSRoundConditionalGains

open MeasureTheory ProbabilityTheory InformationTheory
open scoped ENNReal BigOperators
Formal statement
namespace BanditAlgorithm

/-- Posterior diagonal representation in the conditional proof of the
Thompson-sampling information ratio. -/
theorem bayesian_ts_conditional_diagonal_representation
    {k n : ℕ} [NeZero k]
    (Q : Measure (Fin n → Fin k → ℝ)) [IsProbabilityMeasure Q]
    (hQ : ∀ᵐ X ∂Q, ∀ (s : Fin n) (a : Fin k), X s a ∈ Set.Icc (0 : ℝ) 1)
    {pi : BanditPolicy k} (hpi : IsBayesianTSPolicy Q pi) (t : Fin n) :
    let history := fun p : (Fin n → Fin k → ℝ) × BanditHistory k n ↦
      fun s : Fin t.1 ↦ p.2 (Fin.castLE (Nat.le_of_lt t.2) s)
    let nu := Measure.map history (bayesianAdversarialMeasure Q pi n le_rfl)
    ∀ᵐ h ∂nu, ∃ (p : Fin k → ℝ)
      (P M : Fin k → Measure (Set.Icc (0 : ℝ) 1)),
      (∀ a, IsProbabilityMeasure (P a)) ∧
      (∀ a, IsProbabilityMeasure (M a)) ∧
      (∀ a, klDiv (P a) (M a) ≠ ∞) ∧
      bayesianTSRoundConditionalRegretGain Q pi t h =
        ∑ a, p a * ((∫ z, z.1 ∂P a) - ∫ z, z.1 ∂M a) ∧
      (∑ a, p a ^ 2 * (klDiv (P a) (M a)).toReal) ≤
        bayesianTSRoundConditionalInformationGain Q pi t h := by
  sorry

end BanditAlgorithm
Source
Lattimore and Szepesvari, Bandit Algorithms (2020), Lemma 36.7, printed p. 470 / free PDF p. 479, especially the posterior action probabilities and diagonal KL terms in the proof.

View graph

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paperProve2Me Team
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me