Proof of Theorem 2, p. 30 — the outer minimisation over λ for logistic regression
ProvedRWPI.Classif.logistic_outer_mindistributionally-robust-optimizationdualitylogistic-regressionp2o-batch-pfp2bp2o-gran-per-chapterp2o-plan-paperp2o-v1
Let satisfy . Let with , let and . Then
Combined with Proposition 1, the left-hand side is the worst-case expected logistic loss over the ball ; the identity is therefore the per- form of the logistic half of Theorem 2.
Formalization Note The paper's chain passes through indicators printed as and ; by the previous step the inner supremum is finite at , so the correct indicators are and . The statement here is the first member of the chain equal to its last member, which is unaffected. Norms are in Theorem 2's convention (the page swaps and ). Values are in .
Preamble
import Mathlib import Definitions.Def_RWPI_Classif_losses open scoped ENNReal
Formal statement
namespace RWPI.Classif
/-- App. A.1, proof of Theorem 2, p. 30 (the chain after the inner supremum): for labels
`Yᵢ ∈ {−1, +1}` and `δ ≥ 0`,
`inf_{λ ≥ 0} { δλ + (1/n) Σᵢ sup_x { log(1 + exp(−Yᵢβᵀx)) − λ‖x − Xᵢ‖_q } }
= (1/n) Σᵢ log(1 + exp(−Yᵢβᵀ Xᵢ)) + δ‖β‖_p`. -/
theorem logistic_outer_min {d n : ℕ} (hn : 0 < n) (X : Fin n → Fin d → ℝ) (Y : Fin n → ℝ)
(hY : ∀ i, Y i = 1 ∨ Y i = -1) (p q : ℝ≥0∞) (hpq : p.HolderConjugate q)
(β : Fin d → ℝ) (δ : ℝ) (hδ : 0 ≤ δ) :
(⨅ (lam : ℝ) (_ : 0 ≤ lam), (ENNReal.ofReal (δ * lam) + (n : ℝ≥0∞)⁻¹ *
∑ i, ⨆ x : Fin d → ℝ, (ENNReal.ofReal (logLoss β (x, Y i)) -
ENNReal.ofReal lam * ENNReal.ofReal ‖WithLp.toLp q (x - X i)‖))) =
ENNReal.ofReal ((1 / (n : ℝ)) * ∑ i, logLoss β (X i, Y i) + δ * ‖WithLp.toLp p β‖) := by sorry
end RWPI.Classif
Source
Blanchet, Kang & Murthy, Robust Wasserstein Profile Inference and Applications to Machine Learning, arXiv:1610.05627v4, App. A.1, proof of Theorem 2, p. 30 (third display)
Human review
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.