Katyusha: The First Direct Acceleration of Stochastic Gradient Methods 2: Without Strong Convexity, Katyusha^ns Reaches Error O((F(x₀)−F(x*))/S² + L‖x₀−x*‖²/(mS²))Research Paper
Many problems in machine learning and statistics are regularized empirical risk minimization: minimize an average of loss terms, one per data point, plus a regularizer such as . When is large, a full gradient costs component gradients, so stochastic gradient methods that touch one per…