Optimizing Static Linear Feedback: Gradient Method III: Gradient Descent with the Hessian Step Size Converges Linearly on Strongly Convex FunctionsResearch Paper
Gradient descent needs a step size, and the classical choices each ask for something the user may not have. A constant step needs the Lipschitz constant of the gradient, which is rarely known and often pessimistic. Backtracking needs repeated function evaluations. The exact line search needs a one-dimensional minimization at every iteration.…