A Single-Loop Algorithm for Decentralized Bilevel Optimization: O(1/K) Stationarity and Consensus Rates of SLDBO without Heterogeneity AssumptionsResearch Paper
Bilevel optimization chooses an upper-level variable to minimize an objective that depends on the solution of a lower-level problem. Hyperparameter tuning and meta-learning are the standard examples (Franceschi et al. 2018; Ghadimi and Wang 2018). In the decentralized version the data are split among agents. Each agent sees only its own upper-…