§4.1, p. 345, proof of Lemma 1 — E(e^{−yF(x)}|x) = P(y=1|x)e^{−F(x)} + P(y=−1|x)e^{F(x)}, and J(F) = E_x[E(e^{−yF(x)}|x)]
ProvedAddLogReg.ExpCrit.cond_criterionboostingconditional-expectationexponential-lossp2o-batch-pfp2ap2o-gran-per-chapterp2o-plan-paperp2o-v1
Let be the joint law of on , with marginal on , and let be the conditional class probabilities. Then:
- for every and every real , the conditional exponential criterion at is
- for every measurable , the exponential criterion is the -expectation of the conditional criterion:
This is the first display of the proof of Lemma 1, together with the reduction "it is sufficient to minimize the criterion conditional on ": the joint minimization of splits into one scalar minimization for each .
Formalization Note Both sides of item 2 are lower Lebesgue integrals in ; the conditional criterion is a finite nonnegative real converted with ENNReal.ofReal. Item 1 holds at every , since the conditional kernel is a probability measure at every point.
Preamble
import Mathlib import Definitions.Def_AddLogReg_ExpCrit_Setting open MeasureTheory ProbabilityTheory
Formal statement
namespace AddLogReg.ExpCrit
/-- Proof of Lemma 1, first display (p. 345): `E(e^{−yF(x)}|x) = P(y = 1|x)e^{−F(x)} +
P(y = −1|x)e^{F(x)}`, and the joint criterion is the expectation over `x` of the conditional one. -/
theorem cond_criterion {X : Type*} [MeasurableSpace X] (ν : Measure (X × Bool))
[IsProbabilityMeasure ν] :
(∀ (x : X) (t : ℝ),
condCrit ν x t = condProb ν true x * Real.exp (-t) + condProb ν false x * Real.exp t) ∧
(∀ F : X → ℝ, Measurable F →
J ν F = ∫⁻ x, ENNReal.ofReal (condCrit ν x (F x)) ∂ν.fst) := by sorry
end AddLogReg.ExpCrit
Source
Friedman, Hastie and Tibshirani, Additive logistic regression: a statistical view of boosting, Ann. Statist. 28 (2000), p. 345, §4.1, proof of Lemma 1, first display
Human review
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.