Globally Convergent Type-I Anderson Acceleration for Nonsmooth Fixed-Point Iterations: The Stabilized Type-I Anderson Acceleration Converges to a Fixed Point of Every Nonexpansive MapResearch Paper
Many first-order methods in optimization are fixed-point iterations of a nonexpansive map : proximal gradient descent, projected gradient descent, alternating projections, ISTA and Douglas–Rachford splitting (with the conic solver SCS as an instance) all have this form (Zhang, O'Donoghue, Boyd 2020, §4.2). The…