Minimization Methods for Non-Differentiable Functions VII: Geometric Convergence of Subgradient Methods with Space Dilation along the GradientTextbook
The subgradient method for a nonsmooth convex function converges, but slowly: when the level sets of the objective are elongated, the subgradient is nearly orthogonal to the direction towards the minimum, and the method zigzags. For smooth functions the remedy is a change of metric (Newton and quasi-Newton methods); for nonsmooth functions no Hessian exists…