p. 356 — for J = 2 Algorithm 6 is Algorithm 3: centred F, p = e^F/(e^F + e^{−F}), F ← F + ½ E_w(z|x)
ProvedAddLogReg.Multiclass.two_class_reduceslogistic-regressionlogitboostmulticlass-classificationp2o-batch-pfp2ap2o-gran-per-chapterp2o-plan-paperp2o-v1
Take , a probability measure on , a fit and a point with . Then, in the population version of Algorithm 6:
- the model probability of class 1 is the two-class symmetric logistic (30):
- the updated score of class 1 is
which is Algorithm 3's update with class 1 as ; 3. the update stays centred: the new is minus the new .
This makes precise the sentence "Algorithm 6 is a natural generalization of Algorithm 3 for fitting the J-class logistic regression model (40)".
Formalization Note Classes are Fin 2; Algorithm 3's step is written out here, not imported.
Preamble
import Mathlib import Definitions.Def_AddLogReg_Multiclass_Setting open MeasureTheory ProbabilityTheory ENNReal
Formal statement
namespace AddLogReg.Multiclass
theorem two_class_reduces {X : Type*} [MeasurableSpace X]
(ν : Measure (X × Fin 2)) [IsProbabilityMeasure ν] (F : X → Fin 2 → ℝ) (x : X)
(hF : F x 0 + F x 1 = 0) :
softmax (F x) 0 = Real.exp (F x 0) / (Real.exp (F x 0) + Real.exp (-F x 0)) ∧
logitBoostJStep ν F x 0 =
F x 0 + (1 / 2) * wCondExpBy ν (fun x' _ => classWeight F x' 0)
(fun x' y => workingResponse F x' y 0) x ∧
logitBoostJStep ν F x 1 = -logitBoostJStep ν F x 0 := by sorry
end AddLogReg.Multiclass
Source
Friedman, Hastie and Tibshirani, Additive logistic regression: a statistical view of boosting, Ann. Statist. 28 (2000), p. 356, sentence before Result 6; p. 351, Algorithm 3 and (30)
Human review
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.