Lemma 27.4 (Dudley's chaining): R(A) ≤ c2^{−M}/√m + (6c/m) ∑_{k=1}^M 2^{−k} √(log N(c2^{−k}, A)) for any enclosing radius c
ProvedUnderstandingML.dudley_chainingchainingcovering-numbersmassart-lemmarademacher-complexity
Lemma 27.4. Let . Then, for any integer ,
Formally: for any center and any with on (the book's minimal is the special case), nonempty, ; covering numbers of the bounded set are finite.
Preamble
import Definitions.Def_UnderstandingML_Covering open MeasureTheory
Formal statement
namespace UnderstandingML
/-- **Lemma 27.4 (Dudley's chaining)** (p. 389). Let `c = min_ā max_{a ∈ A} ‖a − ā‖`. Then, for
any integer `M > 0`,
`R(A) ≤ c 2^{−M}/√m + (6c/m) ∑_{k=1}^M 2^{−k} √(log N(c 2^{−k}, A))`.
Stated for any center `ā` and any `c` with `‖a − ā‖ ≤ c` on `A` (the minimal such `c` is the
book's); `A` nonempty, `m ≥ 1`. -/
theorem dudley_chaining {m : ℕ} (hm : 0 < m) (A : Set (Fin m → ℝ)) (hA : A.Nonempty) (c : ℝ)
(abar : Fin m → ℝ) (hc : ∀ a ∈ A, eucNorm (a - abar) ≤ c) (M : ℕ) (hM : 0 < M) :
rademacher A ≤ c * (2 : ℝ)⁻¹ ^ M / Real.sqrt m +
6 * c / m * ∑ k ∈ Finset.Icc 1 M,
(2 : ℝ)⁻¹ ^ k * Real.sqrt (Real.log ((coveringNumber (c * (2 : ℝ)⁻¹ ^ k) A).toNat)) := by sorry
end UnderstandingML
Source
Shalev-Shwartz and Ben-David, Understanding Machine Learning: From Theory to Algorithms, Cambridge University Press 2014, doi:10.1017/CBO9781107298019, §27.2 pp. 389-390, Lemma 27.4 with its proof
Human review
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.