Prove2Me
Navigate
DiscoverCollectionsFormalpediaGalaxyBlogsUsersMomentumMy Missions+
Prove2Me
Log in
← Formalpedia

The two UCB failure counts in Eq. (8.4)

Definition
asymptoticUcbFailureCount

by Harry_Xu · Jul 26, 2026 · Mathlib c5ea003 (Lean v4.30.0)

banditsprobabilityucb

Fix a stochastic bandit ν\nuν, an optimal arm aaa, a target arm iii, a tolerance ε\varepsilonε, and a length-nnn history. After all arms are initialized, let UnU_nUn​ count rounds where the optimal-arm UCB is at most μ∗−ε\mu^*-\varepsilonμ∗−ε, and let VnV_nVn​ count rounds where arm iii is selected while its UCB is at least μ∗−ε\mu^*-\varepsilonμ∗−ε. asymptoticUcbFailureCount returns (Un,Vn)(U_n,V_n)(Un​,Vn​), the two counts in Eq. (8.4).

Definition code
import Definitions.Def_asymptoticUcbPolicy
import Definitions.Def_banditHistoryPrefix

/-!
Lattimore--Szepesvári, *Bandit Algorithms* (CUP 2020), Theorem 8.1,
Eq. (8.4), printed pp. 119--120 / PDF pp. 128--129.

For a fixed optimal arm `a` and suboptimal arm `i`, Eq. (8.4) splits pulls
of `i` after initialization into two bad-index counts: underestimation of the
optimal arm and selection of `i` while its index crosses `μ* - ε`.
The explicit all-arms-initialized condition records the initialization branch
of `IsAsymptoticUCBPolicy`; it only restricts the two source counts.
-/

namespace BanditAlgorithm

/-- The pair of bad-index counts used in Eq. (8.4): optimal-arm
underestimation and selected-suboptimal-arm overshoot. -/
noncomputable def asymptoticUcbFailureCount {k n : ℕ}
    (ν : StochasticBandit k) (a i : Fin k) (ε : ℝ)
    (h : BanditHistory k n) : ℝ × ℝ :=
  let initialized := fun r : Fin n ↦
    ∀ j, armPullCount j (banditHistoryPrefixAt h r) ≠ 0
  let lowOptimal := ∑ r : Fin n,
    if initialized r ∧
        asymptoticUcbIndex a (banditHistoryPrefixAt h r) ≤
          banditOptimalMean ν - ε then (1 : ℝ) else 0
  let highSelected := ∑ r : Fin n,
    if initialized r ∧
        banditOptimalMean ν - ε ≤
          asymptoticUcbIndex i (banditHistoryPrefixAt h r) ∧
        (h r).1 = i then (1 : ℝ) else 0
  (lowOptimal, highSelected)

end BanditAlgorithm
Source
Lattimore and Szepesvári, Bandit Algorithms (CUP 2020), proof of Theorem 8.1, Eq. (8.4), printed pp. 119–120 / PDF pp. 128–129.

View graph

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paperProve2Me Team
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me