(4.14) — under (4.8), : is a descent direction
ProvedDynSampleSize.Det.eq_4_14gradient-methodsinexact-gradientp2o-batch-pfp2bp2o-gran-per-chapterp2o-plan-paperp2o-v1
Let and let satisfy the relative-error condition (4.8), . Then
With the approximate gradient and , this says that is a descent direction for at , with a quantified angle condition. It is the key input of the one-step decrease (4.15)–(4.16).
Formalization Note The statement is a fact about two arbitrary vectors of EuclideanSpace ℝ (Fin m); is the Euclidean inner product inner ℝ v g.
Preamble
import Mathlib import Definitions.Def_DynSampleSize_Det_Setting open Filter Topology
Formal statement
namespace DynSampleSize.Det
theorem eq_4_14 {m : ℕ} (θ : ℝ) (hθ0 : 0 < θ) (hθ1 : θ < 1)
(g v : EuclideanSpace ℝ (Fin m)) (h48 : ‖g - v‖ ≤ θ * ‖g‖) :
(1 - θ) * ‖g‖ ^ 2 ≤ inner ℝ v g := by sorry
end DynSampleSize.Det
Source
Byrd, Chin, Nocedal, Wu, Sample size selection in optimization methods for machine learning, Math. Program. (2012), doi:10.1007/s10107-012-0572-5 — authors' version of 18 Oct 2011, p. 9, (4.14)
Human review
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.