Proposition 2 — DR linear regression with the squared cost: worst-case square loss
ProvedRWPI.SqrtLasso.proposition_2Fix and let satisfy . Let , , be training data with empirical distribution , and let . Take the square loss and the cost on , with optimal transport cost as in (7). Write and . Then:
- for every ,
- consequently (Eq. (13)),
The distributionally robust least-squares problem with an -transport ball is a regularized regression whose penalty is , which also charges the intercept-like coordinate . Theorem 1 removes that coordinate by forbidding transport of the response.
Formalization Note Item 1 is the display that concludes the paper's proof (p. 29); item 2 is the printed statement (13). The page's statement does not mention ; it is the radius of the ball. The data space is , and the cost is the norm of the stacked vector in . Both sides are compared in ; the right side is a real number embedded by ENNReal.ofReal.
import Mathlib import Definitions.Def_WassersteinDRO_Regularization_empiricalDistribution import Definitions.Def_RWPI_SqrtLasso_worstCase import Definitions.Def_RWPI_SqrtLasso_squareLoss import Definitions.Def_RWPI_SqrtLasso_MSE import Definitions.Def_RWPI_SqrtLasso_lqSqCost open MeasureTheory
namespace RWPI.SqrtLasso
/-- Proposition 2 (p. 10), with the pointwise identity its proof establishes (p. 29). Fix
`q ∈ (1, ∞]` and `p` with `1/p + 1/q = 1`. For the square loss and the cost
`c((x, y), (u, v)) = ‖(x, y) − (u, v)‖_q²` on `ℝ^{d+1}`, and every `δ ≥ 0`:
(1) for every `β`, `sup_{P : D_c(P, P_n) ≤ δ} E_P[(Y − βᵀX)²] = (√MSE_n(β) + √δ ‖β̄‖_p)²`;
(2) (13): the infima over `β ∈ ℝ^d` of both sides agree. -/
theorem proposition_2 {d n : ℕ} (hn : 0 < n) (X : Fin n → Fin d → ℝ) (Y : Fin n → ℝ)
(q p : ENNReal) (hq : 1 < q) (hpq : p.HolderConjugate q) (δ : ℝ) (hδ : 0 ≤ δ) :
(∀ β : Fin d → ℝ,
worstCase (lqSqCost q) δ
(WassersteinDRO.Regularization.empiricalDistribution (fun i : Fin n => (X i, Y i)))
(squareLoss β) =
ENNReal.ofReal
((Real.sqrt (MSE X Y β) + Real.sqrt δ * ‖WithLp.toLp p (betaBar β)‖) ^ 2)) ∧
(⨅ β : Fin d → ℝ, worstCase (lqSqCost q) δ
(WassersteinDRO.Regularization.empiricalDistribution (fun i : Fin n => (X i, Y i)))
(squareLoss β)) =
⨅ β : Fin d → ℝ, ENNReal.ofReal
((Real.sqrt (MSE X Y β) + Real.sqrt δ * ‖WithLp.toLp p (betaBar β)‖) ^ 2) := by sorry
end RWPI.SqrtLasso
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.