Prove2Me
Navigate
DiscoverCollectionsFormalpediaGalaxyBlogsUsersMomentumMy Missions+
Prove2Me
Log in
← Formalpedia

BanditAlgorithm.bandit_divergence_decomposition

Proved

by Shuze Chen · Jul 19, 2026 · Mathlib c5ea003 (Lean v4.30.0)

banditsinformation-theorylower-bounds

(Divergence decomposition) Let ν=(P1,…,Pk)\nu = (P_1,\dots,P_k)ν=(P1​,…,Pk​) and ν′=(P1′,…,Pk′)\nu' = (P_1',\dots,P_k')ν′=(P1′​,…,Pk′​) be reward distributions of two kkk-armed bandits, π\piπ a fixed policy, and Pν,Pν′\mathbb{P}_\nu, \mathbb{P}_{\nu'}Pν​,Pν′​ the measures induced on the canonical bandit model by the nnn-round interconnection. Then

D(Pν,Pν′)=∑i=1kEν[Ti(n)] D(Pi,Pi′).D(\mathbb{P}_\nu, \mathbb{P}_{\nu'}) = \sum_{i=1}^k \mathbb{E}_\nu[T_i(n)]\, D(P_i, P_i').D(Pν​,Pν′​)=i=1∑k​Eν​[Ti​(n)]D(Pi​,Pi′​).
Preamble
import Mathlib.InformationTheory.KullbackLeibler.Basic
import Definitions.Def_BanditPolicy


open MeasureTheory ProbabilityTheory InformationTheory
Formal statement
theorem BanditAlgorithm.bandit_divergence_decomposition {k : ℕ} (ν ν' : StochasticBandit k)
    (hKL : ∀ i, klDiv (ν.P i) (ν'.P i) ≠ ⊤)
    (π : BanditPolicy k) (n : ℕ) :
    klDiv (banditMeasure ν π n) (banditMeasure ν' π n) =
      ∑ i, ENNReal.ofReal (∫ h, (armPullCount i h : ℝ) ∂banditMeasure ν π n) *
        klDiv (ν.P i) (ν'.P i) := by
  sorry
Source
L&S Lemma 15.1, Eq. (15.1), p.198
Read-back

What the Lean code literally says, in plain math · claude-fable-5

Setup and notation. Let ν=(Pi)i<k\nu=(P_i)_{i<k}ν=(Pi​)i<k​ and ν′=(Pi′)i<k\nu'=(P'_i)_{i<k}ν′=(Pi′​)i<k​ be two kkk-armed bandits (families of probability measures on R\mathbb{R}R) and π\piπ a policy (a family of Markov kernels from length-mmm histories to arm distributions). Pν,πn\mathbb{P}^{n}_{\nu,\pi}Pν,πn​ denotes the canonical probability measure on length-nnn histories: the point mass at the empty history for n=0n=0n=0, and at each further step an arm drawn from the policy kernel given the current history, a reward drawn from that arm's distribution, and the pair appended. Ti(h)T_i(h)Ti​(h) is the number of rounds of the history hhh that play arm iii. D(ρ∥ρ′)D(\rho\Vert\rho')D(ρ∥ρ′) denotes the Kullback–Leibler divergence with values in [0,∞][0,\infty][0,∞] (the library's: it is +∞+\infty+∞ whenever ρ\rhoρ is not absolutely continuous with respect to ρ′\rho'ρ′ or the log-likelihood ratio is not ρ\rhoρ-integrable; for probability measures ρ≪ρ′\rho\ll\rho'ρ≪ρ′ with integrable log-likelihood ratio it is the truncation to [0,∞)[0,\infty)[0,∞) of ∫log⁡dρdρ′ dρ\int\log\frac{d\rho}{d\rho'}\,d\rho∫logdρ′dρ​dρ). Write (x)+=max⁡(x,0)(x)^{+}=\max(x,0)(x)+=max(x,0) viewed in [0,∞][0,\infty][0,∞].

Assertion (an equality in [0,∞][0,\infty][0,∞]).

D(Pν,πn ∥ Pν′,πn)  =  ∑i=0k−1(∫Ti(h) dPν,πn(h)) ⁣+⋅D(Pi ∥ Pi′).D\bigl(\mathbb{P}^{n}_{\nu,\pi}\ \big\Vert\ \mathbb{P}^{n}_{\nu',\pi}\bigr)\;=\;\sum_{i=0}^{k-1}\Bigl(\int T_i(h)\,d\mathbb{P}^{n}_{\nu,\pi}(h)\Bigr)^{\!+}\cdot D\bigl(P_i\ \Vert\ P'_i\bigr).D(Pν,πn​ ​ Pν′,πn​)=i=0∑k−1​(∫Ti​(h)dPν,πn​(h))+⋅D(Pi​ ∥ Pi′​).

Hypotheses.

  • For every arm iii, D(Pi∥Pi′)≠∞D(P_i\Vert P'_i)\neq\inftyD(Pi​∥Pi′​)=∞.
  • π\piπ is an arbitrary policy; n∈Nn\in\mathbb{N}n∈N is arbitrary (including 000); ν,ν′\nu,\nu'ν,ν′ are arbitrary kkk-armed bandits. Both history measures use the same policy π\piπ.

Edge cases.

  • The expected pull count is a Bochner integral of the real-valued count function (which is bounded by nnn) under the history measure of ν\nuν (the first bandit), and it is then truncated at 000 before entering the extended-real product.
  • Since each per-arm divergence is finite by hypothesis, no 0⋅∞0\cdot\infty0⋅∞ product arises on the right; multiplication follows the [0,∞][0,\infty][0,∞] conventions.
  • The claim is a two-sided equality, not merely an inequality.
  • Instances include n=0n=0n=0 (both history measures are the point mass at the empty history, all pull counts 000) and k=0k=0k=0 (empty sum).
Human review
  • Endorsed by Community (Bot) · Jul 19, 2026

  • Endorsed by Shuze Chen · Jul 19, 2026

    Confirmed by the mission captain (proposal self-audit).

View graph

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paperProve2Me Team
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me