Theorem 5.3, p. 320 — from ‖x₀ − x*‖ ≤ μ/(2M), Newton's method is well defined and ‖x_{k+1} − x*‖ ≤ (M/μ)‖x_k − x*‖²
OpenConvexOptAlg.Newton.theorem_5_3Let be a function whose Hessian is -Lipschitz in operator norm, for all , with . Let be a local minimum of with strictly positive Hessian, for some . Suppose the starting point satisfies
Then Newton's method started at is well defined: there is exactly one sequence with first term satisfying for all , and along it every Hessian is invertible, so that . Moreover the iterates converge to at a quadratic rate:
This is the classical local quadratic convergence of Newton's method: close enough to a nondegenerate local minimum, the number of correct digits roughly doubles at every step. It is the starting point of the interior point methods of §5.3, which keep Newton's method inside such a region of fast convergence.
Formalization Note is EuclideanSpace ℝ (Fin n); the gradient and Hessian are explicit maps tied to (definition item), and the Hessian condition at is the quadratic-form inequality . "Well-defined" is formalized as unique existence of a Newton run from together with invertibility (bijectivity) of every Hessian along every such run; the quadratic rate and the convergence are asserted for every Newton run from . is a disclosed implicit hypothesis (the radius divides by ; with Lean's would force ). Convexity of is not assumed, as on the page.
import Mathlib import Definitions.Def_ConvexOptAlg_Newton_Defs
namespace ConvexOptAlg.Newton
open Filter Topology
/-- **Theorem 5.3** (Bubeck, arXiv:1405.4980v2, p. 320). Let `f : ℝⁿ → ℝ` be C² with gradient map
`g` and Hessian map `H`, and assume the Hessian is `M`-Lipschitz in operator norm. Let `x∗` be a
local minimum of `f` with `∇²f(x∗) ⪰ μ Iₙ`, `μ > 0`, i.e. `μ‖v‖² ≤ ⟪∇²f(x∗) v, v⟫` for all `v`.
If `‖x₀ − x∗‖ ≤ μ/(2M)`, then Newton's method started at `x₀` is well-defined — there is exactly one
Newton run from `x₀`, and along every Newton run from `x₀` each Hessian `∇²f(x_k)` is invertible —
and it converges to `x∗` at a quadratic rate: `‖x_{k+1} − x∗‖ ≤ (M/μ)‖x_k − x∗‖²` for every `k ≥ 0`.
`0 < M` is a disclosed implicit hypothesis (the radius `μ/(2M)` divides by `M`). -/
theorem theorem_5_3 {n : ℕ} (f : EuclideanSpace ℝ (Fin n) → ℝ)
(g : EuclideanSpace ℝ (Fin n) → EuclideanSpace ℝ (Fin n))
(H : EuclideanSpace ℝ (Fin n) → (EuclideanSpace ℝ (Fin n) →L[ℝ] EuclideanSpace ℝ (Fin n)))
(hfgH : IsC2GradHess f g H) (M μ : ℝ) (hM : 0 < M) (hHL : IsLipschitzHessian H M)
(xstar : EuclideanSpace ℝ (Fin n)) (hmin : IsLocalMin f xstar) (hμ : 0 < μ)
(hHstar : ∀ v : EuclideanSpace ℝ (Fin n), μ * ‖v‖ ^ 2 ≤ inner ℝ (H xstar v) v)
(x0 : EuclideanSpace ℝ (Fin n)) (hx0 : ‖x0 - xstar‖ ≤ μ / (2 * M)) :
(∃! x : ℕ → EuclideanSpace ℝ (Fin n), x 0 = x0 ∧ IsNewtonRun g H x) ∧
∀ x : ℕ → EuclideanSpace ℝ (Fin n), x 0 = x0 → IsNewtonRun g H x →
(∀ k, Function.Bijective (H (x k))) ∧
(∀ k, ‖x (k + 1) - xstar‖ ≤ M / μ * ‖x k - xstar‖ ^ 2) ∧
Tendsto x atTop (𝓝 xstar) := by sorry
end ConvexOptAlg.Newton