Prove2Me
Navigate
DiscoverCollectionsFormalpediaGalaxyBlogsUsersMomentumMy Missions+
Prove2Me
Log in
← Formalpedia

§3.2, p. 4 — if ℓ(w,Z) is σ-subgaussian for every w, then each empirical risk L_S(w) is σ/√n-subgaussian under μ^{⊗n}

Proved
InfoGen.Expected.empRisk_subgaussian

by mikedeng1 · Oct 8, 2026 · Mathlib 0df444a (Lean v4.33.1)

empirical-riskiidinformation-theoretic-generalizationp2o-batch-pfp2ap2o-gran-per-chapterp2o-plan-paperp2o-v1subgaussian

Let Z\mathsf ZZ be an instance space, W\mathsf WW a hypothesis space, ℓ:W×Z→R+\ell:\mathsf W\times\mathsf Z\to\mathbb R_+ℓ:W×Z→R+​ a nonnegative, jointly measurable loss, and μ\muμ a probability measure on Z\mathsf ZZ. Let n≥1n\ge 1n≥1 and let S=(Z1,…,Zn)∼μ⊗nS=(Z_1,\dots,Z_n)\sim\mu^{\otimes n}S=(Z1​,…,Zn​)∼μ⊗n be an i.i.d. sample. Write Lμ(w)=E[ℓ(w,Z)]L_\mu(w)=\mathbb E[\ell(w,Z)]Lμ​(w)=E[ℓ(w,Z)] for the population risk and LS(w)=1n∑i=1nℓ(w,Zi)L_S(w)=\frac1n\sum_{i=1}^n\ell(w,Z_i)LS​(w)=n1​∑i=1n​ℓ(w,Zi​) for the empirical risk.

Suppose that for every w∈Ww\in\mathsf Ww∈W the random variable ℓ(w,Z)\ell(w,Z)ℓ(w,Z), Z∼μZ\sim\muZ∼μ, is σ\sigmaσ-subgaussian: log⁡E[eλ(ℓ(w,Z)−Lμ(w))]≤λ2σ2/2\log\mathbb E[e^{\lambda(\ell(w,Z)-L_\mu(w))}]\le\lambda^2\sigma^2/2logE[eλ(ℓ(w,Z)−Lμ​(w))]≤λ2σ2/2 for all λ∈R\lambda\in\mathbb Rλ∈R. Then for every w∈Ww\in\mathsf Ww∈W the empirical risk LS(w)L_S(w)LS​(w) is σ/n\sigma/\sqrt nσ/n​-subgaussian under μ⊗n\mu^{\otimes n}μ⊗n:

log⁡E[eλ (LS(w)−Lμ(w))]≤λ2σ22nfor all λ∈R.\log\mathbb E\Big[e^{\lambda\,(L_S(w)-L_\mu(w))}\Big]\le\frac{\lambda^2\sigma^2}{2n}\qquad\text{for all }\lambda\in\mathbb R .logE[eλ(LS​(w)−Lμ​(w))]≤2nλ2σ2​for all λ∈R.

This is the first half of the step that connects Lemma 1 to the learning problem: with f(s,w)=Ls(w)f(s,w)=L_s(w)f(s,w)=Ls​(w), each section f(S,w)f(S,w)f(S,w) inherits subgaussianity from the loss, with the variance proxy reduced by the factor nnn because of the i.i.d. sample.

Formalization Note Subgaussianity is Mathlib's HasSubgaussianMGF of the centred variable, whose parameter is the variance proxy: σ2\sigma^2σ2 for the loss, σ2/n\sigma^2/nσ2/n for the empirical risk. Joint measurability of ℓ\ellℓ and n≥1n\ge1n≥1 are added (the page assumes both implicitly). The page's nonnegativity of ℓ\ellℓ is kept as a hypothesis although it plays no role.

Preamble
import Mathlib
import Definitions.Def_InfoGen_Expected_Setting

open MeasureTheory ProbabilityTheory InformationTheory
open scoped ENNReal NNReal
Formal statement
namespace InfoGen.Expected

open LearnStability.Characterization

/-- §3.2, p. 4, first clause (Xu & Raginsky, arXiv:1705.07809v2). If `ℓ(w, Z)` is
`σ`-subgaussian under `μ` for every hypothesis `w`, then for every `w` the empirical risk
`L_S(w) = f(S, w)` of an i.i.d. sample `S ∼ μ^{⊗n}` is `σ/√n`-subgaussian: its centred version
`L_S(w) − L_μ(w)` has variance proxy `σ²/n`. -/
theorem empRisk_subgaussian {Z W : Type*} [MeasurableSpace Z] [MeasurableSpace W]
    (μ : Measure Z) [IsProbabilityMeasure μ] (n : ℕ) (hn : 0 < n)
    (ℓ : W → Z → ℝ) (hℓ : Measurable (Function.uncurry ℓ)) (hℓ0 : ∀ w z, 0 ≤ ℓ w z)
    (σ : ℝ≥0) (hσ : ∀ w, HasSubgaussianMGF (fun z => ℓ w z - risk ℓ μ w) (σ ^ 2) μ) :
    ∀ w, HasSubgaussianMGF (fun s => empRisk ℓ s w - risk ℓ μ w) (σ ^ 2 / n)
      (sampleLaw μ n) := by sorry

end InfoGen.Expected
Source
Xu & Raginsky, arXiv:1705.07809v2, §3.2, p. 4, first sentence ("f(S, w) is σ/√n-subgaussian due to the i.i.d. assumption on Z_i's"), with f(s, w) defined on p. 3
Human review
  • Endorsed by Shuze Chen · Oct 8, 2026

    Confirmed by the moderator at approval.

  • Endorsed by mikedeng1 · Oct 8, 2026

    Confirmed by the mission captain (proposal self-audit).

View graph

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paperProve2Me Team
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me