(36), p. 352 — the Newton update f(x) solves the weighted least-squares problem min E_w(F + ½z − (F + f))²
ProvedAddLogReg.LogitBoost.eq_36boostinglogistic-regressionlogitboostp2o-batch-pfp2ap2o-gran-per-chapterp2o-plan-paperp2o-v1weighted-least-squares
Let be a probability measure on , , , the working response and the weight of Algorithm 3. Let
be the Newton update of (34). Then solves the weighted least-squares approximation about to the log-likelihood,
that is, for every real ,
and is the population LogitBoost update of Algorithm 3.
Formalization Note The page writes the weighted expectation in (36) as without ""; since the Derivation conditions on throughout and depends on only, it is read as the conditional weighted expectation .
Preamble
import Mathlib import Definitions.Def_AddLogReg_LogitBoost_Setting open MeasureTheory ProbabilityTheory
Formal statement
namespace AddLogReg.LogitBoost
theorem eq_36 {X : Type*} [MeasurableSpace X] (ν : Measure (X × Bool)) [IsProbabilityMeasure ν]
(F : X → ℝ) (x : X) :
(∀ t : ℝ,
wCondExpBy ν (logitWeight F)
(fun x' b => (F x' + (1 / 2) * workingResponse F x' b -
(F x' + -((2 * ∫ b', (ystar b' - symLogistic (F x)) ∂(ν.condKernel x)) /
(-4 * ∫ _b', symLogistic (F x) * (1 - symLogistic (F x)) ∂(ν.condKernel x))))) ^ 2) x ≤
wCondExpBy ν (logitWeight F)
(fun x' b => (F x' + (1 / 2) * workingResponse F x' b - (F x' + t)) ^ 2) x) ∧
F x + -((2 * ∫ b, (ystar b - symLogistic (F x)) ∂(ν.condKernel x)) /
(-4 * ∫ _b, symLogistic (F x) * (1 - symLogistic (F x)) ∂(ν.condKernel x))) =
logitBoostStep ν F x := by sorry
end AddLogReg.LogitBoost
Source
Friedman, Hastie and Tibshirani, Additive logistic regression: a statistical view of boosting, Ann. Statist. 28 (2000), p. 352, (36)
Human review
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.