Prove2Me
Navigate
DiscoverCollectionsFormalpediaGalaxyBlogsUsersMomentumMy Missions+
Prove2Me
Log in
← Formalpedia

Exponential-weights regret with the exact Ψq\Psi_qΨq​ stability term

Proved
BanditAlgorithm.expWeights_psi_regret

by Harry_Xu · Aug 5, 2026 · Mathlib 0df444a (Lean v4.33.1)

exponential-weightsonline-learningpartial-monitoring

Let k≥1k\ge1k≥1, let η>0\eta>0η>0, and let y^t∈Rk\widehat y_t\in\mathbb R^ky​t​∈Rk be an arbitrary sequence. Define exponential-weights normalizers and distributions by

Wt=∑a=1kexp⁡ ⁣(−η∑s<ty^sa),Qta=exp⁡ ⁣(−η∑s<ty^sa)Wt.W_t=\sum_{a=1}^k\exp\!\left(-\eta\sum_{s<t}\widehat y_{sa}\right), \qquad Q_{ta}=\frac{\exp\!\left(-\eta\sum_{s<t}\widehat y_{sa}\right)}{W_t}.Wt​=a=1∑k​exp(−ηs<t∑​y​sa​),Qta​=Wt​exp(−η∑s<t​y​sa​)​.

Then for every comparator action a0a_0a0​,

∑t=0n−1∑a=1kQta(y^ta−y^ta0)≤log⁡kη+1η∑t=0n−1ΨQt(ηy^t),\sum_{t=0}^{n-1}\sum_{a=1}^k Q_{ta}(\widehat y_{ta}-\widehat y_{ta_0}) \le \frac{\log k}{\eta} +\frac1\eta\sum_{t=0}^{n-1}\Psi_{Q_t}(\eta\widehat y_t),t=0∑n−1​a=1∑k​Qta​(y​ta​−y​ta0​​)≤ηlogk​+η1​t=0∑n−1​ΨQt​​(ηy​t​),

where

Ψq(z)=∑aqa(e−za+za−1).\Psi_q(z)=\sum_a q_a\bigl(e^{-z_a}+z_a-1\bigr).Ψq​(z)=a∑​qa​(e−za​+za​−1).

No boundedness or sign condition is imposed on the estimated-loss vectors. This is the deterministic exponential-weights inequality used by Algorithm 26 before taking expectations.

Preamble
import Mathlib.Analysis.SpecialFunctions.Exp
import Mathlib.Analysis.SpecialFunctions.Log.Basic
import Mathlib.Data.Fintype.Order

open scoped BigOperators

namespace BanditAlgorithm
Formal statement
theorem expWeights_psi_regret
    (k n : ℕ) (hk : 1 ≤ k) (η : ℝ) (hη : 0 < η)
    (y : ℕ → Fin k → ℝ) :
    let W := fun t : ℕ =>
      ∑ a : Fin k, Real.exp (- (η * ∑ s ∈ Finset.range t, y s a))
    let Q := fun t : ℕ => fun a : Fin k =>
      Real.exp (- (η * ∑ s ∈ Finset.range t, y s a)) / W t
    ∀ a₀ : Fin k,
      (∑ t ∈ Finset.range n, ∑ a : Fin k, Q t a * (y t a - y t a₀)) ≤
        Real.log k / η + (1 / η) *
          ∑ t ∈ Finset.range n, ∑ a : Fin k,
            Q t a * (Real.exp (-η * y t a) + η * y t a - 1) := by sorry
Source
Lattimore and Szepesvári, Bandit Algorithms (2020), equation (37.11), printed p. 493, used in the proof of Theorem 37.15 on pp. 494–495, https://tor-lattimore.com/downloads/book/book.pdf

View graph

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paperProve2Me Team
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me