Theorem 7 — the f-divergence robust objective converges uniformly to E_{P₀}[ℓ(x; ξ)] almost surely
ProvedGenEmpLik.Consistency.theorem_7Let be i.i.d. random elements of a separable metric space with law , let be the empirical distribution of the first , and let be nonempty and closed. Let be lower semicontinuous on and measurable for each . Let satisfy Assumption A, let , let Assumption E hold, and assume the class is Glivenko–Cantelli. Then
Thus the robust objective, and every reweighting in the divergence ball, converges to the population risk uniformly over the decision set, under only slightly more than the first-moment condition needed for sample average approximation.
Formalization Note A distribution is a weight vector on the first samples (0-based), feasible when probUncertaintySet f (1/n,…,1/n) (ρ/n). Both suprema are taken in . Almost-sure convergence is ∀ᵐ (outer measure of the exceptional set is zero), the same reading as in the Glivenko–Cantelli hypothesis. i.i.d. means measurable, mutually independent (iIndepFun) and identically distributed with ; is the law of .
import Mathlib import Definitions.Def_PhiDivRobust_Counterpart_uncertaintySet import Definitions.Def_GenEmpLik_Coverage_AssumptionA import Definitions.Def_GenEmpLik_Consistency_AssumptionE import Definitions.Def_GenEmpLik_Consistency_IsGlivenkoCantelli open MeasureTheory ProbabilityTheory Filter Topology open scoped ENNReal
namespace GenEmpLik.Consistency
/-- Theorem 7 (arXiv:1610.03425v3, p. 16; proof App. E.1, pp. 46–47). Let `ξ₀, ξ₁, …` be
i.i.d. with law `P₀` on a separable metric space, let `f` satisfy Assumption A, `ρ ≥ 0`,
and let `X` be nonempty and closed. Let the lower semicontinuous loss `ℓ` satisfy Assumption E
on `X`, with measurable `ℓ(x; ·)` for `x ∈ X`, and let `{ℓ(x; ·) : x ∈ X}` be
Glivenko–Cantelli. Then, almost surely,
`sup_{x ∈ X} sup { |E_P[ℓ(x; ξ)] − E_{P₀}[ℓ(x; ξ)]| : P ≪ P̂_n, D_f(P ‖ P̂_n) ≤ ρ/n } → 0`,
with `P ≪ P̂_n` the weight vectors `p` on the first `n` samples and the suprema in `[0, ∞]`. -/
theorem theorem_7 {Ω Ξ : Type*} [MeasurableSpace Ω] [MeasurableSpace Ξ] [MetricSpace Ξ]
[SecondCountableTopology Ξ]
{μ : Measure Ω} [IsProbabilityMeasure μ] {d : ℕ}
(f : ℝ → EReal) (hf : GenEmpLik.Coverage.AssumptionA f) (ρ : ℝ) (hρ : 0 ≤ ρ)
(ξ : ℕ → Ω → Ξ) (hξm : ∀ i, Measurable (ξ i)) (hξind : iIndepFun ξ μ)
(hξid : ∀ i, IdentDistrib (ξ i) (ξ 0) μ μ)
(X : Set (EuclideanSpace ℝ (Fin d))) (hXne : X.Nonempty) (hXclosed : IsClosed X)
(ℓ : EuclideanSpace ℝ (Fin d) → Ξ → ℝ)
(hℓlsc : LowerSemicontinuous (fun xs : X × Ξ => ℓ xs.1 xs.2))
(hℓm : ∀ x ∈ X, Measurable (ℓ x))
(hE : AssumptionE (μ.map (ξ 0)) X ℓ)
(hGC : IsGlivenkoCantelli μ ξ ((fun x => ℓ x) '' X)) :
∀ᵐ ω ∂μ, Tendsto (fun n : ℕ => ⨆ x ∈ X,
⨆ p ∈ PhiDivRobust.Counterpart.probUncertaintySet f (fun _ : Fin n => (1 : ℝ) / n)
(ρ / n),
ENNReal.ofReal |∑ i, p i * ℓ x (ξ i ω) - ∫ s, ℓ x s ∂(μ.map (ξ 0))|)
atTop (𝓝 0) := by sorry
end GenEmpLik.Consistency
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.