(34)–(35), p. 352 — F(x) − H(x)⁻¹s(x) = F(x) + ½E(y*−p|x)/E(p(1−p)|x) = F(x) + ½E_w((y*−p)/(p(1−p))|x)
ProvedAddLogReg.LogitBoost.eq_34_35boostinglogistic-regressionlogitboostnewton-methodp2o-batch-pfp2ap2o-gran-per-chapterp2o-plan-paperp2o-v1
Let be a probability measure on , , , and let
be the score (32) and Hessian (33). Then , and the Newton update satisfies
where is the weighted conditional expectation with weight .
The last expression is plus one half of the population weighted least-squares fit of the working response of Algorithm 3, which is what identifies LogitBoost's step with a Newton step.
Formalization Note is part of the conclusion, not a hypothesis: it holds because . Without it the Lean division would silently return .
Preamble
import Mathlib import Definitions.Def_AddLogReg_LogitBoost_Setting open MeasureTheory ProbabilityTheory
Formal statement
namespace AddLogReg.LogitBoost
theorem eq_34_35 {X : Type*} [MeasurableSpace X] (ν : Measure (X × Bool))
[IsProbabilityMeasure ν] (F : X → ℝ) (x : X) :
(-4 * ∫ _b, symLogistic (F x) * (1 - symLogistic (F x)) ∂(ν.condKernel x) ≠ 0) ∧
F x - (2 * ∫ b, (ystar b - symLogistic (F x)) ∂(ν.condKernel x)) /
(-4 * ∫ _b, symLogistic (F x) * (1 - symLogistic (F x)) ∂(ν.condKernel x)) =
F x + (1 / 2) * ((∫ b, (ystar b - symLogistic (F x)) ∂(ν.condKernel x)) /
∫ _b, symLogistic (F x) * (1 - symLogistic (F x)) ∂(ν.condKernel x)) ∧
F x + (1 / 2) * ((∫ b, (ystar b - symLogistic (F x)) ∂(ν.condKernel x)) /
∫ _b, symLogistic (F x) * (1 - symLogistic (F x)) ∂(ν.condKernel x)) =
F x + (1 / 2) * wCondExpBy ν (logitWeight F) (workingResponse F) x := by sorry
end AddLogReg.LogitBoost
Source
Friedman, Hastie and Tibshirani, Additive logistic regression: a statistical view of boosting, Ann. Statist. 28 (2000), p. 352, (34)–(35)
Human review
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.