(37), p. 354 — E(e^{−yF}y|x)/E(e^{−yF}|x) = (P − p(x))/((1 − p(x))P + p(x)(1 − P)), always in [−1, 1]
ProvedAddLogReg.Gentle.eq_37boostinggentle-adaboostlogistic-regressionlogitboostp2o-batch-pfp2ap2o-gran-per-chapterp2o-plan-paperp2o-v1
Let be a probability measure on , the joint law of , let and fix . Let and let
be the symmetric logistic of . Then
- this common value always lies in .
Written in terms of and , the Gentle AdaBoost update can be compared directly with the LogitBoost update (38).
Formalization Note is not assumed; it follows from the conditional law being a probability measure. The symmetric logistic is the exact expression above, not , with which (37) is false.
Preamble
import Mathlib import Definitions.Def_AddLogReg_Gentle_Setting open MeasureTheory ProbabilityTheory
Formal statement
namespace AddLogReg.Gentle
theorem eq_37 {X : Type*} [MeasurableSpace X] (ν : Measure (X × Bool))
[IsProbabilityMeasure ν] (F : X → ℝ) (x : X) :
(∫ b, Real.exp (-(AddLogReg.ExpCrit.sgn b * F x)) * AddLogReg.ExpCrit.sgn b ∂(ν.condKernel x)) /
(∫ b, Real.exp (-(AddLogReg.ExpCrit.sgn b * F x)) ∂(ν.condKernel x)) =
(Real.exp (-F x) * AddLogReg.ExpCrit.condProb ν true x - Real.exp (F x) * (1 - AddLogReg.ExpCrit.condProb ν true x)) /
(Real.exp (-F x) * AddLogReg.ExpCrit.condProb ν true x + Real.exp (F x) * (1 - AddLogReg.ExpCrit.condProb ν true x)) ∧
(Real.exp (-F x) * AddLogReg.ExpCrit.condProb ν true x - Real.exp (F x) * (1 - AddLogReg.ExpCrit.condProb ν true x)) /
(Real.exp (-F x) * AddLogReg.ExpCrit.condProb ν true x + Real.exp (F x) * (1 - AddLogReg.ExpCrit.condProb ν true x)) =
(AddLogReg.ExpCrit.condProb ν true x - AddLogReg.LogitBoost.symLogistic (F x)) /
((1 - AddLogReg.LogitBoost.symLogistic (F x)) * AddLogReg.ExpCrit.condProb ν true x +
AddLogReg.LogitBoost.symLogistic (F x) * (1 - AddLogReg.ExpCrit.condProb ν true x)) ∧
(AddLogReg.ExpCrit.condProb ν true x - AddLogReg.LogitBoost.symLogistic (F x)) /
((1 - AddLogReg.LogitBoost.symLogistic (F x)) * AddLogReg.ExpCrit.condProb ν true x +
AddLogReg.LogitBoost.symLogistic (F x) * (1 - AddLogReg.ExpCrit.condProb ν true x)) ∈ Set.Icc (-1 : ℝ) 1 := by sorry
end AddLogReg.Gentle
Source
Friedman, Hastie and Tibshirani, Additive logistic regression: a statistical view of boosting, Ann. Statist. 28 (2000), p. 354, §4.4, (37) and the paragraph after (38)
Human review
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.