(7.4): the regularized consensus -update is averaging followed by a proximal step with weight
ProvedBoydADMM.Consensus.regularized_z_update_proxadmmconsensusp2o-batch-pfp2bp2o-gran-per-chapterp2o-plan-bookp2o-v1proximal-operator
Let , , and let be any regularizer. Fix (standing for ) and (standing for ). Then a point minimizes the -update objective (7.4)
over if and only if it minimizes the proximal objective
over .
So the central collector first averages and then applies the proximal operator of with parameter .
Formalization Note Each extended-real-valued function of the book is encoded by its effective domain (a set) and its finite values on it (a real function); a minimization of a sum containing is a minimization over . No convexity of is needed: the two objectives differ by a constant independent of . The minimizer sets are compared, not a selected minimizer.
Preamble
import Mathlib import Definitions.Def_BoydADMM_Consensus_Model open scoped BigOperators InnerProductSpace
Formal statement
namespace BoydADMM.Consensus
/-- §7.1.1, p. 52: for a regularizer `g` with effective domain `Cg`, the `z`-update (7.4)
`argmin_z (g(z) + ∑_i (−y_i^{kT} z + (ρ/2)‖x_i^{k+1} − z‖²))` has exactly the same
minimizers as the averaging-then-proximal step
`argmin_z (g(z) + (Nρ/2)‖z − x̄^{k+1} − (1/ρ)ȳ^k‖²)`. Here `x`, `y` stand for `x^{k+1}`, `y^k`. -/
theorem regularized_z_update_prox {n N : ℕ} (hN : 0 < N) (ρ : ℝ) (hρ : 0 < ρ)
(Cg : Set (EuclideanSpace ℝ (Fin n))) (g : EuclideanSpace ℝ (Fin n) → ℝ)
(x y : Fin N → EuclideanSpace ℝ (Fin n)) (z : EuclideanSpace ℝ (Fin n)) :
(z ∈ Cg ∧ ∀ z' ∈ Cg,
g z + ∑ i, (-⟪y i, z⟫_ℝ + (ρ / 2) * ‖x i - z‖ ^ 2) ≤
g z' + ∑ i, (-⟪y i, z'⟫_ℝ + (ρ / 2) * ‖x i - z'‖ ^ 2)) ↔
(z ∈ Cg ∧ ∀ z' ∈ Cg,
g z + ((N : ℝ) * ρ / 2) * ‖z - avg x - (1 / ρ) • avg y‖ ^ 2 ≤
g z' + ((N : ℝ) * ρ / 2) * ‖z' - avg x - (1 / ρ) • avg y‖ ^ 2) := by sorry
end BoydADMM.Consensus
Source
Boyd, Parikh, Chu, Peleato, Eckstein, Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers, Found. Trends Mach. Learn. 3(1) (2011), p. 52, §7.1.1, (7.4) and the averaging-plus-proximal form of the z-update
Human review
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.