Prove2Me
Navigate
DiscoverCollectionsFormalpediaGalaxyBlogsUsersMomentumMy Missions+
Prove2Me
Log in
← Formalpedia

KL-UCB pathwise feasibility-count split

Proved
BanditAlgorithm.bandit_kl_ucb_feasibility_pull_count_split

by Harry_Xu · Jul 29, 2026 · Mathlib c5ea003 (Lean v4.30.0)

banditskl-ucbprobability

Consider KL-UCB on a Bernoulli bandit with arm means μj∈[0,1]\mu_j\in[0,1]μj​∈[0,1]. Fix an optimal arm aaa, a suboptimal arm iii, and a tolerance 0<ε<Δi0<\varepsilon<\Delta_i0<ε<Δi​, where Δi=μ⋆−μi\Delta_i=\mu^\star-\mu_iΔi​=μ⋆−μi​. Let Ui(n)U_i(n)Ui​(n) be the number of times arm iii is selected before horizon nnn. Let La,i,ε(n)L_{a,i,\varepsilon}(n)La,i,ε​(n) count selections of iii caused by incomplete initialization or failure of the optimal arm's KL threshold test, and let Vi,ε(n)V_{i,\varepsilon}(n)Vi,ε​(n) count initialized selections of iii for which the selected arm passes its KL threshold test.

Then the expected pull count is bounded by the expectations of these two failure counts:

E[Ui(n)]≤E[La,i,ε(n)]+E[Vi,ε(n)].\mathbb E[U_i(n)] \le \mathbb E[L_{a,i,\varepsilon}(n)] + \mathbb E[V_{i,\varepsilon}(n)].E[Ui​(n)]≤E[La,i,ε​(n)]+E[Vi,ε​(n)].

Indeed, on every realized Bernoulli reward history, if KL-UCB selects iii, maximality of its index implies a dichotomy. Either the optimal arm does not regard μ⋆−ε\mu^\star-\varepsilonμ⋆−ε as feasible, or the selected arm does. Thus every selection of iii is charged pathwise to exactly one of the two displayed counts, and integration gives the result.

This lemma is the deterministic-probabilistic interface in the proof of the finite-time KL-UCB regret bound: subsequent concentration lemmas bound the two expectations separately.

Formalization Note: The policy hypothesis is expressed by IsKLUCBPolicy; the Bernoulli-bandit equality supplies the almost-sure {0,1}\{0,1\}{0,1}-valued reward support needed to place empirical means in [0,1][0,1][0,1]. The Lean proof first establishes the inequality on every supported finite history, then verifies measurability and integrability of all three finite counts before applying monotonicity of the integral.

Preamble
import Definitions.Def_klucbFeasibilityFailureCount

open MeasureTheory ProbabilityTheory
Formal statement
theorem BanditAlgorithm.bandit_kl_ucb_feasibility_pull_count_split
    {k : ℕ} (μvec : Fin k → ℝ)
    (hμ : ∀ i, μvec i ∈ Set.Icc (0 : ℝ) 1)
    (ν : BanditAlgorithm.StochasticBandit k)
    (hν : ν = BanditAlgorithm.bernoulliBandit μvec hμ)
    (π : BanditAlgorithm.BanditPolicy k)
    (hπ : BanditAlgorithm.IsKLUCBPolicy π)
    (n : ℕ) (a i : Fin k) (ε : ℝ)
    (ha : BanditAlgorithm.banditArmMean ν a =
      BanditAlgorithm.banditOptimalMean ν)
    (hε : 0 < ε) (hεgap : ε < BanditAlgorithm.banditGap ν i) :
    MeasureTheory.integral (BanditAlgorithm.banditMeasure ν π n)
        (fun h ↦ (BanditAlgorithm.armPullCount i h : ℝ)) ≤
      MeasureTheory.integral (BanditAlgorithm.banditMeasure ν π n)
          (fun h ↦
            (BanditAlgorithm.klucbFeasibilityFailureCount ν a i ε h).1) +
        MeasureTheory.integral (BanditAlgorithm.banditMeasure ν π n)
          (fun h ↦
            (BanditAlgorithm.klucbFeasibilityFailureCount ν a i ε h).2) := by
  sorry
Source
Lattimore and Szepesvári, Bandit Algorithms (2020), Algorithm 8 and proof of Theorem 10.6, printed pp. 137 and 139–140 (PDF pp. 126 and 128–129).

View graph

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paperProve2Me Team
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me