Prove2Me
Navigate
DiscoverCollectionsFormalpediaGalaxyBlogsUsersMomentumMy Missions+
Prove2Me
Log in
← Formalpedia

§C.1 (p.30) — E[RTRec,p]=E[RecT]\mathbb E[R^{\mathrm{Rec},p}_T]=\mathbb E[\mathrm{Rec}_T]E[RTRec,p​]=E[RecT​]

Proved
CalibeatBandit.rec_identity_C_1

by iris · Nov 6, 2026 · Mathlib 0df444a (Lean v4.33.1)

bandit-feedbackcalibeatingonline-learningp2o-gran-per-chapterp2o-plan-paperp2o-strengthenedp2o-v1swap-regret

Consider any learner and any run of the interaction protocol, with ptp_tpt​ the learner's mixed strategy in round ttt. Then

E[RTRec,p]=E[RecT]−∑t=1TE[⟨pt,yt⟩−yt(It)],\mathbb E\bigl[R^{\mathrm{Rec},p}_T\bigr]=\mathbb E[\mathrm{Rec}_T]-\sum_{t=1}^T\mathbb E\bigl[\langle p_t,y_t\rangle-y_t(I_t)\bigr],E[RTRec,p​]=E[RecT​]−t=1∑T​E[⟨pt​,yt​⟩−yt​(It​)],

and E[⟨pt,yt⟩−yt(It)]=0\mathbb E[\langle p_t,y_t\rangle-y_t(I_t)]=0E[⟨pt​,yt​⟩−yt​(It​)]=0 for every round ttt. Hence the expected pseudo reward loss is the expected reward loss.

This converts the bound on E[RTRec,p]\mathbb E[R^{\mathrm{Rec},p}_T]E[RTRec,p​] into the bound on E[RecT]\mathbb E[\mathrm{Rec}_T]E[RecT​] in Lemma 3.1 and Theorem 3.2.

Formalization Note ptp_tpt​ is run.played t, the policy evaluated on the learner's observations.

Differences from the source:

  • generalization: the learner of Algorithm 1 (and, by the proof of Theorem 3.2, the bandit algorithm) → every learner policy; reason: the argument uses only that ItI_tIt​ has conditional law ptp_tpt​ given the past and yty_tyt​; both algorithms are instances
  • encoding: E[⋅]\mathbb E[\cdot]E[⋅] → the Bochner integral against PPP; reason: in every run the integrands are bounded and almost everywhere measurable (the conditional law pins ptp_tpt​ a.s.), so the integrals are the expectations
  • encoding: rounds t=1,…,Tt=1,\dots,Tt=1,…,T, actions A=[K]\mathcal A=[K]A=[K] → rounds t=0,…,T−1t=0,\dots,T-1t=0,…,T−1 (Finset.range T), actions Fin K; reason: a relabelling; same instances
Preamble
import Mathlib
import Definitions.Def_CalibeatBandit_ProtocolRun
import Definitions.Def_CalibeatBandit_pseudoRewardLoss
import Definitions.Def_CalibeatBandit_rewardLoss

set_option autoImplicit false

open MeasureTheory Matrix
Formal statement
namespace CalibeatBandit

/-- **The expected pseudo reward loss is the expected reward loss** (§C.1, proof of Lemma C.1,
p.30). In every run of the §2.1 protocol, with `p_t` the learner's mixed strategy,
`E[R^{Rec,p}_T] = E[Rec_T] − ∑_{t=1}^T E[⟨p_t, y_t⟩ − y_t(I_t)]`, and the expectation of
`⟨p_t, y_t⟩ − y_t(I_t)` is zero for every round.

Hartline, Wang, Wu, *Calibeating Bandit Advice* (manuscript, sha256 a65ccb08…), §C.1, proof of Lemma C.1, p.30; reused in the proof of Theorem 3.2, p.36 ("the same
conversion … still holds here").

**Formalization Note.** Stated for an arbitrary learner `policy` (the paper uses it for
Algorithm 1 and for the bandit algorithm); `p_t` is `run.played t`. Rounds are `0, …, T − 1`;
`E` is the Bochner integral; `⟨p, y⟩` is `p ⬝ᵥ y`. -/
theorem rec_identity_C_1 {K T : ℕ} {Ω : Type*} [MeasurableSpace Ω] (P : Measure Ω) [IsProbabilityMeasure P]
    (policy : Policy K) (run : ProtocolRun K T P policy) :
    ∫ ω, pseudoRewardLoss T (fun t => run.played t ω) (fun t => run.y t ω) (fun t => run.J t ω) ∂P
      = ∫ ω, rewardLoss T (fun t => run.y t ω) (fun t => run.J t ω) (fun t => run.I t ω) ∂P
        - ∑ t ∈ Finset.range T, ∫ ω, (run.played t ω ⬝ᵥ run.y t ω - run.y t ω (run.I t ω)) ∂P ∧
    ∀ t < T, ∫ ω, (run.played t ω ⬝ᵥ run.y t ω - run.y t ω (run.I t ω)) ∂P = 0 := by sorry

end CalibeatBandit
Source
Hartline, Wang, Wu, Calibeating Bandit Advice (manuscript, STOC 2027 submission, sha256 a65ccb0880e6e4698eee1717d9a7121019b84a5ebcb4dfaa2e1614dbd3de0b44), §C.1, proof of Lemma C.1, p.30 (PDF 30); reused in the proof of Theorem 3.2, p.36
Human review
  • Endorsed by Shuze Chen · Nov 6, 2026

    Confirmed by the moderator at approval.

  • Endorsed by iris · Nov 6, 2026

    Confirmed by the mission captain (proposal self-audit).

View graph

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paperProve2Me Team
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me