Lemma 3.4 — the Bayes risk is the integral of the minimal inner risks (measurable loss)
ProvedSupportVectorMachines.Calibration.lemma_3_4_bayes_risk_via_inner_risk_v2This is Lemma 3.4 (Computation of Bayes risks) of Steinwart & Christmann, Support Vector Machines (Springer 2008, p. 52), the bridge between the outer Bayes risk and the pointwise (inner) minimization apparatus of Chapter 3.
Let be a complete measurable space, a loss (Definition 2.1: measurable and nonnegative), and a distribution on , represented by with a measurable family of conditional distributions. Then is measurable, and
The Bayes risk — an infimum over all measurable functions on — is thus exactly the integral of a pointwise, one-real-variable minimization.
Formalization Note. The retired version quantified over the bare function type X → ℝ → ℝ → ℝ, so a loss depending non-measurably on made non-measurable (the accepted disproof). The corrected statement takes in the bundled Loss X (measurable and nonnegative, Definition 2.1's own requirements) and uses the book's definition of a complete measurable space. Conventions made explicit (common to the corrected Chapter 3 milestones): a loss is the bundled Loss X of Definition 2.1 — measurable on and nonnegative — with the closed label set embedded in (a loss on extends by outside , a distribution on is one on supported on , and a set of distributions on is a set of measures on ); a distribution on is represented by its marginal (a probability measure) together with a measurable family of probability measures, the regular conditional probabilities (which exist since is Polish, Lemma A.3.16; every statement is invariant under the choice of version), so that risks are written in the form of Eq. (3.5); all risks are -valued Lebesgue integrals, as in the book; and IsCompleteMeasurableSpace is now the book's own notion (the -algebra equals its universal completion) instead of the retired stronger sufficient condition.
import Mathlib import Definitions.Def_SupportVectorMachines_Calibration_Loss_v2 import Definitions.Def_SupportVectorMachines_Calibration_InnerRisks_v2 import Definitions.Def_SupportVectorMachines_Calibration_OuterRisks_v2 import Definitions.Def_SupportVectorMachines_Calibration_IsCompleteMeasurableSpace_v2 open MeasureTheory
namespace SupportVectorMachines.Calibration
/-- Lemma 3.4 (Computation of Bayes risks), Steinwart & Christmann, *Support Vector Machines*,
Springer 2008, p. 52: let `X` be a complete measurable space, `L : X × Y × ℝ → [0,∞)` be a loss
(Definition 2.1: measurable and nonnegative, both bundled in `Loss X`), and `P` (represented by
`(PX, κ)`, `κ` a measurable family of conditional distributions `P(·|x)`) be a distribution on
`X × ℝ`. Then `x ↦ C*_{L,P(·|x),x}` is measurable and `R*_{L,P} = ∫_X C*_{L,P(·|x),x} dPX(x)`.
Corrected version of `lemma_3_4_bayes_risk_via_inner_risk`, whose `Loss X` was the bare function
type (no measurability), and whose completeness notion was a stronger sufficient condition. -/
theorem lemma_3_4_bayes_risk_via_inner_risk_v2 {X : Type*} [MeasurableSpace X]
(hX : IsCompleteMeasurableSpace X) (L : Loss X)
(PX : Measure X) [IsProbabilityMeasure PX] (κ : X → Measure ℝ) (hκ : Measurable κ)
(hκprob : ∀ x, IsProbabilityMeasure (κ x)) :
Measurable (fun x => minInnerRisk L (κ x) x) ∧
bayesRisk L PX κ = ∫⁻ x, minInnerRisk L (κ x) x ∂PX := by sorry
end SupportVectorMachines.Calibration