Motivation
Many problems in imaging, signal processing and statistics take the form
x∈Xmin F(x)+G(x)+H(Lx),
where G and H are convex functions whose proximity operators can be computed cheaply, L is a linear operator such as a finite-difference gradient, and F is smooth. Total-variation denoising, the lasso with a structured penalty, and constrained least squares are of this type. Because H∘L is generally not proximable even when H is, practical methods split the problem so that each step uses only proxτG, proxσH∗, L and L∗, without inverting any operator.
L. Condat (J. Optim. Theory Appl. 158 (2013)) introduced a relaxed, inexact primal–dual iteration of this kind; B. C. Vũ (Adv. Comput. Math. 38 (2013)) studied the same structure for monotone inclusions. With F=0 the iteration is exactly the method of Chambolle and Pock (J. Math. Imaging Vis. 40 (2011)). They proved convergence in finite dimension assuming τσ∥L∥2<1, ρn≡1 and no errors. He and Yuan (SIAM J. Imaging Sci. 5 (2012)) extended this to a constant relaxation ρn≡ρ∈]0,2[ under the same other hypotheses (Condat, §3.1.1). Condat's paper proves three convergence theorems. This mission is the third: in finite dimension and with F=0, the iterates converge under the step-size condition στ∥L∥2≤1, equality included. Equality matters in practice: one can set σ=1/(τ∥L∥2) and tune a single parameter, as in the Douglas–Rachford method.
Setting
Let X and Y be real Hilbert spaces and L:X→Y a bounded linear operator with adjoint L∗ and operator norm ∥L∥. Write Γ0(H) for the proper, lower semicontinuous, convex functions H→R∪{+∞}, and let G∈Γ0(X), H∈Γ0(Y). The Fenchel conjugate is H∗(s)=sups′[⟨s,s′⟩−H(s′)], the proximity operator is proxJ(s)=argmins′[J(s′)+21∥s−s′∥2], and the subdifferential is ∂J(u)={v: ⟨u′−u,v⟩+J(u)≤J(u′) ∀u′}.
The primal–dual inclusion (6) asks for (x^,y^)∈X×Y with
0∈∂G(x^)+L∗y^+∇F(x^),0∈−Lx^+∂H∗(y^).
A solution gives a minimiser x^ of the primal problem and a solution y^ of its dual.
Algorithm 3.1 chooses τ>0, σ>0, relaxation parameters (ρn), error terms (eF,n),(eG,n),(eH,n) and an initial estimate (x0,y0), then iterates
x~n+1=proxτG(xn−τ(∇F(xn)+eF,n)−τL∗yn)+eG,n,y~n+1=proxσH∗(yn+σL(2x~n+1−xn))+eH,n,
(xn+1,yn+1)=ρn(x~n+1,y~n+1)+(1−ρn)(xn,yn).
Algorithm 3.2 swaps the roles of the primal and dual variables: the dual step comes first, and the primal step uses 2y~n+1−yn. Section 5 extends both to ∑i=1mHi(Lix) (Algorithms 5.1 and 5.2), with the inclusion (48) in place of (6).
Formalization targets
Goal: Theorem 3.3 (p. 6)
Let X, Y be finite-dimensional, F=0, eF,n=0, and assume (6) has a solution. If
(i) στ∥L∥2≤1,(ii) ρn∈[ε,2−ε] ∀n, for some ε>0,(iii) ∑n∥eG,n∥<∞, ∑n∥eH,n∥<∞,
then for every run of Algorithm 3.1, and for every run of Algorithm 3.2, (xn,yn) converges to a solution (x^,y^) of (6).
Milestones (from the proof, pp. 8–13)
With P(x,y)=(τ1x−L∗y,−Lx+σ1y) the operator (20) and T(x,y)=(x~,y~) the error-free step of Algorithm 3.1:
- P (and P′ of (44)) is positive under (i): ⟨z,Pz⟩≥0;
- T depends on z only through Pz (the paper's T∘S=T, (32)–(33));
- on solutions of (6), PT(z)=Pz ((41)–(42));
- PT(z)=Pz implies that T(z) solves (6) (via (35));
- T is continuous;
- Lemma 4.1 (Krasnosel'skii–Mann iteration) and Lemma 4.6 (Polyak's lemma).
Further statements
Remark 3.2 (Theorem 3.3 with F affine, i.e. β=0 in (2)) and Theorem 5.3 (the analogue for m≥2 composite terms, with (i) replaced by στ∥∑iLi∗Li∥≤1) are included as draft theorems.
Significance
Theorem 3.3 is the convergence guarantee behind the common practice of running the Chambolle–Pock iteration and its relaxed variants at the critical step size στ∥L∥2=1. It covers relaxation parameters up to 2−ε and summable errors in both proximity operators. It applies directly to the discrete models of imaging and statistics, which are finite-dimensional. Theorem 5.3 extends it to any finite number of composite terms in parallel.
None of the statements of this paper is formalized on the platform. Machine-checked convergence proofs for primal–dual splitting are not available in Mathlib. The mission would produce the first ones, together with two standalone tools of general use: the inexact Krasnosel'skii–Mann theorem (Lemma 4.1), and Polyak's recursive-inequality lemma (Lemma 4.6), which is a standard tool for stochastic and inexact iterations.
Difficulty
The usual proof treats the iteration as a proximal-point or forward–backward step in the space X×Y with the inner product ⟨z,Pz′⟩. That argument needs P strictly positive, which is exactly what fails when στ∥L∥2=1: then P has a nontrivial kernel, ⟨z,Pz⟩ is only a seminorm, and weak convergence in the P-geometry says nothing about the components of z in kerP. The proof replaces the iteration by its "shadow" Szn on ranP, uses that T factors through S, and recovers the full iterates through continuity of T and a recursive inequality. The last step requires strong convergence of the shadow sequence, which is where finite dimension enters. Infinite-dimensional versions require different arguments and are not claimed here.
Formalization scope
Spaces are real inner product spaces with CompleteSpace; the goal and Theorem 5.3 add FiniteDimensional. Functions in Γ0 take values in EReal and satisfy the published predicate IsProperClosedConvex (never −∞, finite somewhere, lower semicontinuous, convex epigraph). The conjugate is an EReal supremum. Proximity operators are maps PG, PH satisfying the published minimisation predicate IsProx for τG and σH∗; such maps exist and are unique for Γ0 functions. The subdifferential is the published IsSubgradient. Runs of the algorithms are predicates on pairs of sequences with arbitrary initial point, and the limit is chosen after the run. "=+∞" for a series is divergence of its partial sums, "<+∞" is summability of a nonnegative series, and convergence in the goal is norm convergence.
The standing assumptions of pp. 3–4 are hypotheses: G,H∈Γ0, and (6) has a solution. The paper's other standing assumption, that problem (1) has a minimiser, follows from the second and is omitted. In the milestones, the operators P and T are plain maps on X×Y. The projector S is not built: "T∘S=T" is stated as "Pz=Pz′⇒T(z)=T(z′)", which is equivalent because P is self-adjoint. Each milestone drops finite dimension, so it is stated at least as strongly as on the page.
The strict inequality στ∥L∥2<1 would make the goal a corollary of the weaker Theorem 3.2 with an extra finite-dimensional upgrade. The goal keeps ≤. Weak convergence in place of norm convergence, an ε chosen after n, or a solution of (6) fixed before the run would each weaken the theorem, and all are excluded.
Useful infrastructure includes: firm nonexpansiveness of prox for EReal-valued Γ0 functions; Γ0-ness of the conjugate and Moreau's identity; maximal monotonicity of ∂G×∂H∗ plus a skew operator; and the inexact Krasnosel'skii–Mann theorem. All of it can be reused for Theorems 3.1 and 3.2 of the same paper, and for Douglas–Rachford and three-operator splitting. Proofs of individual milestones are welcome independently of the goal.
Selected references
- L. Condat, A primal–dual splitting method for convex optimization involving Lipschitzian, proximable and linear composite terms, J. Optim. Theory Appl. 158(2):460–479, 2013. https://doi.org/10.1007/s10957-012-0245-9 (author's version: https://hal.science/hal-00609728v5)
- B. C. Vũ, A splitting algorithm for dual monotone inclusions involving cocoercive operators, Adv. Comput. Math. 38:667–681, 2013. https://doi.org/10.1007/s10444-011-9254-8
- A. Chambolle, T. Pock, A first-order primal–dual algorithm for convex problems with applications to imaging, J. Math. Imaging Vis. 40:120–145, 2011. https://doi.org/10.1007/s10851-010-0251-1
- P. L. Combettes, Solving monotone inclusions via compositions of nonexpansive averaged operators, Optimization 53:475–504, 2004. https://doi.org/10.1080/02331930412331327157
- H. H. Bauschke, P. L. Combettes, Convex Analysis and Monotone Operator Theory in Hilbert Spaces, Springer, 2011. https://doi.org/10.1007/978-1-4419-9467-7
- B. T. Polyak, Introduction to Optimization, Optimization Software, New York, 1987.