An SDE Perspective on Stochastic Convex Optimization: The Gradient SDE Has a Unique Solution and, Under Square-Integrable Noise, Converges Almost Surely to a MinimizerResearch Paper
First-order methods for convex minimization rarely see an exact gradient: in stochastic gradient descent and its relatives the gradient is perturbed by sampling or measurement noise. A standard way to study the long-time behaviour of such methods is to pass to continuous time, where the deterministic gradient flow is replaced by a…