Prove2Me
Navigate
DiscoverFormalpediaBlogsUsersMomentumMy Missions+
Prove2Me
⌕
Log in

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

Operations Research

911 missions · 537 completed

The discipline of applying mathematical analysis to complex decision problems in operations: allocating scarce resources, scheduling, routing, inventory, and the design of service and production systems. Drawing on mathematical programming, stochastic modeling, queueing, simulation, and game-theoretic reasoning, it seeks policies that perform provably well in systems shaped by constraints, congestion, and uncertainty.

Missions

Open374Completed537All911
🏆Completed
OptimizationProbability·Captain: mikedeng1

A Distributional Interpretation of Robust Optimization I: Robust Optimization over Overlapping Uncertainty Sets Equals a Distributionally Robust Stochastic ProgramResearch Paper

Motivation

Robust optimization (RO) protects a decision against every realisation of an uncertain parameter in a prescribed uncertainty set; distributionally robust stochastic programming (DRSP) protects it against every probability distribution in a prescribed distribution set. The two paradigms are usually treated separately. When the n uncertain parameters live in different spaces, it is folklore that RO over a product of sets is DRSP over the distributions supported on that product (Delage and Ye, Operations Research 2010).

In data-driven problems the situation is different: the parameters x1,…,xnx_1,\dots,x_nx1​,…,xn​ are samples, and all of them lie in the same space Rm\mathbb R^mRm. Robustifying each sample by its own uncertainty set Zi\mathcal Z_iZi​ gives the objective ∑iciinf⁡xi∈Zif(xi)\sum_i c_i\inf_{x_i\in\mathcal Z_i}f(x_i)∑i​ci​infxi​∈Zi​​f(xi​), and the sets Zi\mathcal Z_iZi​ typically overlap. Xu, Caramanis and Mannor (Math. Oper. Res. 2012) show that this objective is again a worst-case expectation, now over distributions on Rm\mathbb R^mRm itself rather than on Rm×n\mathbb R^{m\times n}Rm×n. This equivalence is what the same paper uses to prove that box-robust sample average optimisation is statistically consistent, and to explain the shrinkage heuristic of RO.

Setting

Let m,n≥1m,n\ge1m,n≥1 and write [1:n]={1,…,n}[1:n]=\{1,\dots,n\}[1:n]={1,…,n}. Let P\mathcal PP be the set of Borel probability measures on Rm\mathbb R^mRm. The data are:

  • a measurable utility f:Rm→Rf:\mathbb R^m\to\mathbb Rf:Rm→R (the decision variable is suppressed);
  • weights c1,…,cn>0c_1,\dots,c_n>0c1​,…,cn​>0 with ∑i=1nci=1\sum_{i=1}^n c_i=1∑i=1n​ci​=1;
  • nonempty Borel uncertainty sets Z1,…,Zn⊆Rm\mathcal Z_1,\dots,\mathcal Z_n\subseteq\mathbb R^mZ1​,…,Zn​⊆Rm, which may intersect or coincide.

For S⊆[1:n]S\subseteq[1:n]S⊆[1:n] write ZS=⋃i∈SZi\mathcal Z_S=\bigcup_{i\in S}\mathcal Z_iZS​=⋃i∈S​Zi​ and N=[1:n]N=[1:n]N=[1:n]. The distribution set is

Pn={μ∈P ∣ ∀S⊆[1:n]: μ(ZS)≥∑i∈Sci}.\mathcal P_n=\Big\{\mu\in\mathcal P\ \Big|\ \forall S\subseteq[1:n]:\ \mu(\mathcal Z_S)\ge\sum_{i\in S}c_i\Big\}.Pn​={μ∈P ​ ∀S⊆[1:n]: μ(ZS​)≥i∈S∑​ci​}.

Each μ∈Pn\mu\in\mathcal P_nμ∈Pn​ must give every union of uncertainty sets at least the total weight of its indices. For μ∈P\mu\in\mathcal Pμ∈P the expectation ∫f dμ\int f\,d\mu∫fdμ is the extended integral ∫f+dμ−∫f−dμ∈[−∞,+∞]\int f^+d\mu-\int f^-d\mu\in[-\infty,+\infty]∫f+dμ−∫f−dμ∈[−∞,+∞].

Formalization targets

Goal: Theorem 2.1 (Eq. (4), pp. 96–97)

∑i=1n[ciinf⁡xi∈Zif(xi)]=inf⁡μ∈Pn∫Rmf(x) dμ(x),\sum_{i=1}^n\Big[c_i\inf_{x_i\in\mathcal Z_i}f(x_i)\Big]=\inf_{\mu\in\mathcal P_n}\int_{\mathbb R^m}f(x)\,d\mu(x),i=1∑n​[ci​xi​∈Zi​inf​f(xi​)]=μ∈Pn​inf​∫Rm​f(x)dμ(x),

as an identity in [−∞,+∞][-\infty,+\infty][−∞,+∞], with no boundedness assumption on fff and no disjointness assumption on the Zi\mathcal Z_iZi​.

Milestones (proof of Theorem 2.1, p. 97)

  1. Every μ∈Pn\mu\in\mathcal P_nμ∈Pn​ satisfies μ(Rm∖ZN)=0\mu(\mathbb R^m\setminus\mathcal Z_N)=0μ(Rm∖ZN​)=0, hence ∫Rmf dμ=∫ZNf dμ\int_{\mathbb R^m}f\,d\mu=\int_{\mathcal Z_N}f\,d\mu∫Rm​fdμ=∫ZN​​fdμ.
  2. Weak duality. With fi=inf⁡Ziff_i=\inf_{\mathcal Z_i}ffi​=infZi​​f finite, every α∈R2n\alpha\in\mathbb R^{2^n}α∈R2n satisfying ∑SαS1(x∈ZS)≤f(x)\sum_S\alpha_S\mathbf 1(x\in\mathcal Z_S)\le f(x)∑S​αS​1(x∈ZS​)≤f(x) on ZN\mathcal Z_NZN​ and αS≥0\alpha_S\ge0αS​≥0 for S≠NS\ne NS=N obeys ∑ici∑SαS1(i∈S)≤∑icifi\sum_i c_i\sum_S\alpha_S\mathbf 1(i\in S)\le\sum_i c_if_i∑i​ci​∑S​αS​1(i∈S)≤∑i​ci​fi​.
  3. The nested dual solution. If f1≥⋯≥fnf_1\ge\dots\ge f_nf1​≥⋯≥fn​, the vector with α{1,…,i}=fi−fi+1\alpha_{\{1,\dots,i\}}=f_i-f_{i+1}α{1,…,i}​=fi​−fi+1​, αN=fn\alpha_N=f_nαN​=fn​ and all other coordinates 000 is feasible and has objective ∑icifi\sum_i c_if_i∑i​ci​fi​.

Further statements

  • For pairwise disjoint Zi\mathcal Z_iZi​: Pn={μ∈P∣μ(Zi)=ci, i=1,…,n}\mathcal P_n=\{\mu\in\mathcal P\mid\mu(\mathcal Z_i)=c_i,\ i=1,\dots,n\}Pn​={μ∈P∣μ(Zi​)=ci​, i=1,…,n} (p. 97).
  • Corollary 2.1 (Eq. (5)): inf⁡x′∈Zf(x′)=inf⁡μ∈P, μ(Z)=1∫f dμ\inf_{x'\in\mathcal Z}f(x')=\inf_{\mu\in\mathcal P,\ \mu(\mathcal Z)=1}\int f\,d\muinfx′∈Z​f(x′)=infμ∈P, μ(Z)=1​∫fdμ.
  • Corollary 5.2 (nested distributions, p. 107): for Z1⊆⋯⊆Zn\mathcal Z_1\subseteq\dots\subseteq\mathcal Z_nZ1​⊆⋯⊆Zn​ and 0=p0<p1<⋯<pn=10=p_0<p_1<\dots<p_n=10=p0​<p1​<⋯<pn​=1,
inf⁡μ∈P, μ(Zi)≥pi ∀i∫f dμ=∑i=1n(pi−pi−1)inf⁡xi∈Zif(xi).\inf_{\mu\in\mathcal P,\ \mu(\mathcal Z_i)\ge p_i\ \forall i}\int f\,d\mu=\sum_{i=1}^n(p_i-p_{i-1})\inf_{x_i\in\mathcal Z_i}f(x_i).μ∈P, μ(Zi​)≥pi​ ∀iinf​∫fdμ=i=1∑n​(pi​−pi−1​)xi​∈Zi​inf​f(xi​).

Significance

The result. Theorem 2.1 turns a robust problem with overlapping uncertainty sets into a distributionally robust one on the original space Rm\mathbb R^mRm. This is what allows distributions in Pn\mathcal P_nPn​ to be compared with the true data-generating distribution as nnn grows: in §3 of the paper a kernel density estimator is shown to lie in Pn\mathcal P_nPn​ for box uncertainty sets, which yields consistency of box-robust sample average optimisation (Theorem 3.1); in §4.2 the nested-distribution form (Corollary 5.2) explains why shrinking an uncertainty set approximates a two-scenario DRSP (Theorem 4.1). The disjoint case recovers the classical product-space equivalence.

Formalizing it. The result is proved in the paper, through the strong duality of a semi-infinite linear program (Isii 1962). It has no machine-checked proof. The mission produces the equivalence as an identity of extended reals, together with a reusable definition of the union-mass distribution set. A proof need not follow the paper's duality route; any correct argument is welcome.

Difficulty

The inequality ≥\ge≥ from the left side is the easy half: point masses ∑iciδxi\sum_ic_i\delta_{x_i}∑i​ci​δxi​​ with xi∈Zix_i\in\mathcal Z_ixi​∈Zi​ belong to Pn\mathcal P_nPn​. The substance is the reverse bound, that no μ∈Pn\mu\in\mathcal P_nμ∈Pn​ can do better than ∑icifi\sum_ic_if_i∑i​ci​fi​. For disjoint sets this is immediate, since μ(Zi)=ci\mu(\mathcal Z_i)=c_iμ(Zi​)=ci​. For overlapping sets a measure may place mass in intersections, and a single point of Zi∩Zj\mathcal Z_i\cap\mathcal Z_jZi​∩Zj​ can serve several indices at once; the constraint family over all 2n2^n2n subsets is what prevents this, and the bound has to exploit the whole family, not the singleton constraints. The paper does this by appeal to semi-infinite LP duality, a theorem that Mathlib does not contain. Measure-theoretic side conditions (unbounded Zi\mathcal Z_iZi​, infinite integrals, infima equal to −∞-\infty−∞) must also be handled rather than assumed away.

Formalization scope

  • Rm\mathbb R^mRm is Fin m → ℝ with its Borel σ\sigmaσ-algebra; no norm is used. Indices 1,…,n1,\dots,n1,…,n are Fin n, subsets are Finset (Fin n), and {1,…,i}\{1,\dots,i\}{1,…,i} is Finset.Iic i.
  • Pn\mathcal P_nPn​ is a Set (Measure (Fin m → ℝ)) whose membership includes IsProbabilityMeasure; the constraint is imposed for every subset, ∅\emptyset∅ and [1:n][1:n][1:n] included.
  • ∫f dμ\int f\,d\mu∫fdμ is expect μ f, defined in EReal as the difference of two lower Lebesgue integrals, ∫f+−∫f−\int f^+-\int f^-∫f+−∫f−. The Bochner integral is not used, because its value 000 on non-integrable functions would falsify Eq. (4). Both sides of Eq. (4) are EReal infima; the left infimum ranges over the nonempty set Zi\mathcal Z_iZi​.
  • Readings of the printed statements. (i) The paper allows fff to take the value −∞-\infty−∞; here fff is real-valued. The excluded case is the one the proof disposes of in its first sentence, where both sides are −∞-\infty−∞. (ii) No boundedness hypothesis is added: when some inf⁡Zif=−∞\inf_{\mathcal Z_i}f=-\inftyinfZi​​f=−∞ both sides are −∞-\infty−∞, and otherwise every μ∈Pn\mu\in\mathcal P_nμ∈Pn​ has a finite negative part. (iii) Corollary 2.1 prints "f:R∪{−∞}f:\mathbb R\cup\{-\infty\}f:R∪{−∞}" without a domain; it is read as f:Rm→Rf:\mathbb R^m\to\mathbb Rf:Rm→R. (iv) Corollary 5.2 is corrected: the paper prints the coefficient (pn−pn−1)(p_n-p_{n-1})(pn​−pn−1​), while its proof sets ci=pi−pi−1c_i=p_i-p_{i-1}ci​=pi​−pi−1​; the printed version is false (for Z1={a}⊆Z2={a,b}\mathcal Z_1=\{a\}\subseteq\mathcal Z_2=\{a,b\}Z1​={a}⊆Z2​={a,b}, f(a)=1f(a)=1f(a)=1, f(b)=0f(b)=0f(b)=0, p1=1/4p_1=1/4p1​=1/4, the left side is 1/41/41/4 and the printed right side 3/43/43/4). The mission states (pi−pi−1)(p_i-p_{i-1})(pi​−pi−1​). (v) The standing hypotheses of Theorem 2.1 (fff measurable, Zi\mathcal Z_iZi​ nonempty Borel) are made explicit in Corollary 5.2.
  • The ordering f1≥⋯≥fnf_1\ge\dots\ge f_nf1​≥⋯≥fn​ is a hypothesis of the nested-dual milestone only, as the proof's "without loss of generality"; the goal does not assume it. Milestones 2 and 3 assume fff bounded below on each Zi\mathcal Z_iZi​ (the proof's first reduction), so that fif_ifi​ is a real number.
  • Ruled out. A Bochner-integral formulation, a restriction to disjoint sets, a distribution set containing non-probability measures, or a set Pn\mathcal P_nPn​ defined by the singleton constraints μ(Zi)≥ci\mu(\mathcal Z_i)\ge c_iμ(Zi​)≥ci​ alone would each change or trivialise the theorem; none is used.
  • Definitions (file Model): the set Pn\mathcal P_nPn​, the extended expectation, dual feasibility, and the nested dual vector. The extended expectation and the union-mass distribution set are reusable beyond this mission. Welcome contributions: a proof through semi-infinite LP duality, a direct measure-theoretic proof (for instance a layer-cake argument for the lower bound), and proofs of the corollaries from the goal.

Selected references

  • H. Xu, C. Caramanis, S. Mannor, A Distributional Interpretation of Robust Optimization, Mathematics of Operations Research 37(1):95–110, 2012. https://doi.org/10.1287/moor.1110.0531
  • E. Delage, Y. Ye, Distributionally Robust Optimization Under Moment Uncertainty with Application to Data-Driven Problems, Operations Research 58(3):595–612, 2010. https://doi.org/10.1287/opre.1090.0741
  • K. Isii, On sharpness of Tchebycheff-type inequalities, Annals of the Institute of Statistical Mathematics 14:185–197, 1962. https://doi.org/10.1007/BF02868641
  • A. Ben-Tal, L. El Ghaoui, A. Nemirovski, Robust Optimization, Princeton University Press, 2009. https://doi.org/10.1515/9781400831050
5 thms2 active usersReviewed
🏆Completed
Discrete GeometryLinear OptimizationOptimization·Captain: mikedeng1

Maximal Lattice-Free Convex Sets in Linear Subspaces II: Minimal Valid Inequalities Come from Maximal Lattice-Free Convex SetsResearch Paper

Cutting planes from lattice-free convex sets

Cutting planes for mixed-integer linear programs are often derived from a few rows of an optimal simplex tableau. Keeping qqq rows for basic integer variables x1,…,xqx_1,\dots,x_qx1​,…,xq​, dropping the nonnegativity of xxx (Gomory's corner polyhedron, Gomory 1969) and then the integrality of the nonbasic variables leaves the set of s≥0s\ge0s≥0 with f+∑jrjsj∈Zqf+\sum_j r^js_j\in\mathbb Z^qf+∑j​rjsj​∈Zq. Balas observed in 1971 that convex sets with no integral point in their interior give valid inequalities for such sets (Balas 1971). Andersen, Louveaux, Weismantel and Wolsey (2007) for two rows, and Borozan and Cornuéjols (2009) for any number of rows, showed that for rational data the irredundant valid inequalities correspond to maximal lattice-free convex sets. Basu, Conforti, Cornuéjols and Zambelli (arXiv:1701.06543; Math. Oper. Res. 35(3), 2010) removed the rationality assumption. This mission formalizes that result, Theorem 3 of their paper.

Setting

Fix q≥0q\ge0q≥0, a point f∈Rqf\in\mathbb R^qf∈Rq and a linear subspace W⊆RqW\subseteq\mathbb R^qW⊆Rq, and assume that the affine space f+Wf+Wf+W contains an integral point. Products such as ryryry are standard inner products.

  • W\mathcal WW is the space of real functions s=(sr)r∈Ws=(s_r)_{r\in W}s=(sr​)r∈W​ with finite support. The semi-infinite relaxation is
Rf(W)={s∈W ∣ f+∑r∈Wrsr∈Zq, sr≥0 (r∈W)}.R_f(W)=\Big\{s\in\mathcal W \ \Big|\ f+\sum_{r\in W}rs_r\in\mathbb Z^q,\ s_r\ge0\ (r\in W)\Big\}.Rf​(W)={s∈W ​ f+r∈W∑​rsr​∈Zq, sr​≥0 (r∈W)}.
  • A linear inequality is a pair (ψ,α)(\psi,\alpha)(ψ,α) with ψ:W→R\psi:W\to\mathbb Rψ:W→R an arbitrary function and α∈R\alpha\in\mathbb Rα∈R, read as Ψ(s)=∑r∈Wψ(r)sr≥α\Psi(s)=\sum_{r\in W}\psi(r)s_r\ge\alphaΨ(s)=∑r∈W​ψ(r)sr​≥α. It is valid if every s∈Rf(W)s\in R_f(W)s∈Rf​(W) satisfies it.
  • VVV is the affine hull of (f+W)∩Zq(f+W)\cap\mathbb Z^q(f+W)∩Zq, and V={s∈W∣f+∑rrsr∈V}\mathcal V=\{s\in\mathcal W\mid f+\sum_r rs_r\in V\}V={s∈W∣f+∑r​rsr​∈V}. A linear inequality is trivial if every s∈Vs\in\mathcal Vs∈V with s≥0s\ge0s≥0 satisfies it. When WWW is irrational (not spanned by the integral points it contains, up to translation), VVV is a proper affine subspace of f+Wf+Wf+W.
  • ∑ψ(r)sr≥α\sum\psi(r)s_r\ge\alpha∑ψ(r)sr​≥α dominates ∑ψ′(r)sr≥α\sum\psi'(r)s_r\ge\alpha∑ψ′(r)sr​≥α if ψ≤ψ′\psi\le\psi'ψ≤ψ′ pointwise. A valid inequality is minimal if no valid inequality with the same α\alphaα and a different ψ′≤ψ\psi'\le\psiψ′≤ψ exists.
  • Choose C∈Rℓ×qC\in\mathbb R^{\ell\times q}C∈Rℓ×q and d∈Rℓd\in\mathbb R^\elld∈Rℓ with V={x∈f+W∣Cx=d}V=\{x\in f+W\mid Cx=d\}V={x∈f+W∣Cx=d}. Two valid inequalities are equivalent if ψ(r)=ρψ′(r)+λTCr\psi(r)=\rho\psi'(r)+\lambda^TCrψ(r)=ρψ′(r)+λTCr for all r∈Wr\in Wr∈W and α=ρα′+λT(d−Cf)\alpha=\rho\alpha'+\lambda^T(d-Cf)α=ρα′+λT(d−Cf), for some ρ>0\rho>0ρ>0, λ∈Rℓ\lambda\in\mathbb R^\ellλ∈Rℓ.
  • A maximal lattice-free convex set in f+Wf+Wf+W is a convex B⊆f+WB\subseteq f+WB⊆f+W with no integral point in its interior relative to f+Wf+Wf+W, inclusionwise maximal with these properties.
  • For K⊆WK\subseteq WK⊆W closed, convex, with 000 in its interior relative to WWW: the polar K∗={y∈W∣ry≤1 ∀r∈K}K^*=\{y\in W\mid ry\le1\ \forall r\in K\}K∗={y∈W∣ry≤1 ∀r∈K}, K^={y∈K∗∣∃x∈K, xy=1}\hat K=\{y\in K^*\mid\exists x\in K,\ xy=1\}K^={y∈K∗∣∃x∈K, xy=1}, and ρK(r)=sup⁡y∈K^ry\rho_K(r)=\sup_{y\in\hat K}ryρK​(r)=supy∈K^​ry. For such a BBB with fff in its interior, ψB=ρB−f\psi_B=\rho_{B-f}ψB​=ρB−f​; for a polyhedral B={x∈f+W∣ai(x−f)≤1}B=\{x\in f+W\mid a_i(x-f)\le1\}B={x∈f+W∣ai​(x−f)≤1} with tight rows this is ψB(r)=max⁡iair\psi_B(r)=\max_ia_irψB​(r)=maxi​ai​r.

A function σ:W→R\sigma:W\to\mathbb Rσ:W→R is sublinear if σ(λr)=λσ(r)\sigma(\lambda r)=\lambda\sigma(r)σ(λr)=λσ(r) for λ≥0\lambda\ge0λ≥0 and σ(r+r′)≤σ(r)+σ(r′)\sigma(r+r')\le\sigma(r)+\sigma(r')σ(r+r′)≤σ(r)+σ(r′).

Formalization targets

Goal: Theorem 3 (p. 6)

  1. Every nontrivial valid linear inequality for Rf(W)R_f(W)Rf​(W) is dominated by a nontrivial minimal valid linear inequality for Rf(W)R_f(W)Rf​(W).
  2. Every nontrivial minimal valid linear inequality for Rf(W)R_f(W)Rf​(W) is equivalent to one of the form
∑r∈WψB(r)sr ≥ 1\sum_{r\in W}\psi_B(r)s_r\ \ge\ 1r∈W∑​ψB​(r)sr​ ≥ 1

with ψB≥0\psi_B\ge0ψB​≥0 on WWW and BBB a maximal lattice-free convex set in f+Wf+Wf+W with fff in its interior.

The goal is stated as one conjunction. It fixes no constants and makes no rationality assumption on fff or WWW.

Milestones

In the order in which the proof on pp. 15–21 uses them:

  • Lemma 23 (a sublinear valid inequality below any valid one)
  • Lemma 26 (invariance under equivalence)
  • Claims 1 and 2 in the proof of Theorem 3
  • Theorem 28 (Basu–Cornuéjols–Zambelli: ρK\rho_KρK​ is the smallest sublinear function with 111-sublevel set KKK)
  • Remark 30 and Claim 4 (the inequality ∑ρK(r)sr≥1\sum\rho_K(r)s_r\ge1∑ρK​(r)sr​≥1)
  • Remark 29 (ρK=max⁡iair\rho_K=\max_ia_irρK​=maxi​ai​r for tight rows)
  • Claim 6 (a shift by λTC\lambda^TCλTC making ψ\psiψ nonnegative)
  • Claim 7 (ψB\psi_BψB​ below a nonnegative sublinear ψ′′\psi''ψ′′)
  • Lemma 31 (maximal lattice-free sets give minimal inequalities)

Significance

Theorem 3 says that the minimal valid inequalities of Rf(W)R_f(W)Rf​(W) are exactly those produced by maximal lattice-free convex sets, up to the equivalence forced by the affine hull V\mathcal VV. It also says that for irrational WWW, where valid inequalities can have negative coefficients, some equivalent form always has nonnegative coefficients. The paper derives two further results from it: a description of the closure of conv⁡(Rf(W))\operatorname{conv}(R_f(W))conv(Rf​(W)) in a suitable norm (Theorem 4), and a reduction of extreme inequalities of the infinite model to finite ones (Theorem 5). Both results remain unproved without it.

The result has a complete proof in the paper, which relies on the cited Theorem 28 from Basu, Cornuéjols, Zambelli. To our knowledge none of it is machine-checked. A formalization would provide a definitional layer for corner relaxations, lattice-free sets and valid inequalities, which is currently absent from Mathlib. It would also expose where the page is imprecise; see the scope section.

Difficulty

For rational data every valid inequality can be written with right-hand side 111 and nonnegative coefficients, and ψ\psiψ is then the gauge of BψB_\psiBψ​. For irrational WWW this fails. Since Rf(W)⊆VR_f(W)\subseteq\mathcal VRf​(W)⊆V, adding λTCr\lambda^TCrλTCr to ψ\psiψ changes nothing on Rf(W)R_f(W)Rf​(W), so coefficients can be negative, and Bψ={x∈f+W∣ψ(x−f)≤α}B_\psi=\{x\in f+W\mid\psi(x-f)\le\alpha\}Bψ​={x∈f+W∣ψ(x−f)≤α} may have a full-dimensional recession cone. A maximal lattice-free set containing BψB_\psiBψ​ then yields a ψB\psi_BψB​ that need not lie below ψ\psiψ (the example on pp. 21–22 shows this). One must first pass to an equivalent inequality whose set has no full-dimensional recession cone within VVV. This step combines the structure theorem for maximal lattice-free sets in irrational subspaces (Theorem 9 of the paper) with a duality argument. A second obstacle is that ψ\psiψ is an arbitrary function: validity alone gives no convexity or continuity, and Lemma 23 is needed to recover them.

Formalization scope

  • Ambient space. Rq\mathbb R^qRq is EuclideanSpace ℝ (Fin q), WWW is a Submodule, and W\mathcal WW is W →₀ ℝ. The printed phrase "the set {r∣sr>0}\{r\mid s_r>0\}{r∣sr​>0} has finite cardinality" is read as ordinary finite support.
  • Standing hypothesis. Every statement assumes that f+Wf+Wf+W contains an integral point. Without it Rf(W)=∅R_f(W)=\emptysetRf​(W)=∅ and every inequality is valid, so this hypothesis rules out the trivializing formalization. The equivalence predicate carries the hypothesis V={x∈f+W∣Cx=d}V=\{x\in f+W\mid Cx=d\}V={x∈f+W∣Cx=d} together with validity of both inequalities. With C,dC,dC,d unconstrained, equivalence would be rescaling only, and part 2 of the goal would be false for irrational WWW.
  • Interiors. All interiors are relative: to f+Wf+Wf+W for BBB and BψB_\psiBψ​, to WWW for KKK.
  • ψB\psi_BψB​. It is defined as ρB−f\rho_{B-f}ρB−f​, independent of any description of BBB.

The paper is imprecise in three places, and the formalization departs from the page in each:

  1. Remark 29 is false without tight rows: for K=(−∞,1]⊆RK=(-\infty,1]\subseteq\mathbb RK=(−∞,1]⊆R written with a1=1a_1=1a1​=1, a2=1/2a_2=1/2a2​=1/2, ρK(r)=r≠max⁡(r,r/2)\rho_K(r)=r\ne\max(r,r/2)ρK​(r)=r=max(r,r/2) for r<0r<0r<0. It is stated with the tightness the paper arranges before using it.
  2. The identity int⁡(Bψ)={x∣ψ(x−f)<α}\operatorname{int}(B_\psi)=\{x\mid\psi(x-f)<\alpha\}int(Bψ​)={x∣ψ(x−f)<α} on p. 17 fails at α=0\alpha=0α=0, for example for ψ≡0\psi\equiv0ψ≡0. Claims 1 and 2 use the strict sublevel set, and Claim 2 is false for the topological interior.
  3. Claims 5 and 7 invoke Corollary 20 in f+Wf+Wf+W, although it is proved only for a lattice of a linear space. Corollary 20 and Claim 5 are therefore not milestones.

Two kinds of contributions are especially welcome: reusable infrastructure for polars of convex sets relative to a subspace and for sublinear functions on submodules, and a proof of Theorem 28.

Selected references

  • A. Basu, M. Conforti, G. Cornuéjols, G. Zambelli, Maximal lattice-free convex sets in linear subspaces, Math. Oper. Res. 35(3), 2010; arXiv:1701.06543v1. https://arxiv.org/abs/1701.06543v1
  • A. Basu, G. Cornuéjols, G. Zambelli, Convex sets and minimal sublinear functions, J. Convex Anal. 18(2), 2011 (reference [9]).
  • V. Borozan, G. Cornuéjols, Minimal valid inequalities for integer constraints, Math. Oper. Res. 34(3), 2009. https://doi.org/10.1287/moor.1090.0400
  • K. Andersen, Q. Louveaux, R. Weismantel, L. Wolsey, Inequalities from two rows of a simplex tableau, IPCO 2007. https://doi.org/10.1007/978-3-540-72792-7_1
  • E. Balas, Intersection cuts — a new type of cutting planes for integer programming, Oper. Res. 19, 1971. https://doi.org/10.1287/opre.19.1.19
  • R. E. Gomory, Some polyhedra related to combinatorial problems, Linear Algebra Appl. 2, 1969. https://doi.org/10.1016/0024-3795(69)90017-2
16 thms2 active usersReviewed
🏆Completed
ProbabilityStochastic Systems·Captain: Shuze Chen

Processing Networks VI: Feedforward and Generalized Jackson Network StabilityTextbook

Motivation

Mission V supplied the general Lyapunov machinery — the extinction criteria of Lemmas 8.5, 8.6 and 8.11 — but a Lyapunov function does not construct itself. For a specific network structure and control policy, one must exhibit a concrete function and verify the drift condition. J. G. Dai and J. Michael Harrison's Processing Networks: Fluid Models and Stability (Cambridge University Press, forthcoming; cited here from the authors' pre-publication draft, 2020-4-2, http://spnbook.org) devotes the second half of Chapter 8 to two such constructions, chosen to illustrate the two basic templates every later stability chapter follows: a piecewise-linear Lyapunov function tailored to a structural network property (feedforward routing), and a linear one that works for an unrestricted network but is tailored to a specific control policy (head-of-line proportional service, HLSPS). Together they cover, as a corollary, the generalized Jackson network — the classical multiclass queueing network with one server per class — under ordinary non-idling FCFS.

Setting

A queueing network (Section 2.6, restated from mission IV) is feedforward (Definition 8.13) if its stations admit a numbering under which no station routes work to a lower-numbered one (feedback to the same station is still allowed). The workload operator W(z):=AM(I−P′)−1zW(z) := AM(I-P')^{-1}zW(z):=AM(I−P′)−1z (Eq. 8.24) gives, for any buffer-contents vector zzz, the total service effort each pool would need to drain zzz to emptiness with no further arrivals; the load vector of the standard load condition is ρ=W(λ)\rho = W(\lambda)ρ=W(λ) (Eq. 8.25), and the total arrival rate vector α\alphaα (including internally routed traffic) is the unique solution of the traffic equations α=λ+P′α\alpha = \lambda + P'\alphaα=λ+P′α. A queueing network under HLSPS control (Definition 8.17) splits each pool's capacity among its classes in fixed proportions γi:=αimi/ρp(i)\gamma_i := \alpha_i m_i/\rho_{p(i)}γi​:=αi​mi​/ρp(i)​ (Eq. 8.30) — a policy general enough to reduce, for a generalized Jackson network (one class per pool), to ordinary non-idling FCFS.

Formalization targets

Goal: Theorem 8.18 — HLSPS control is stable under the standard load condition

For any queueing network under HLSPS control with proportion vector γ\gammaγ, if

ρ<b(the standard load condition, Eq. 5.1),\rho < b \qquad (\text{the standard load condition, Eq. 5.1}),ρ<b(the standard load condition, Eq. 5.1),

the corresponding fluid model is stable, and hence (Theorem 6.2) the network itself is stable under HLSPS control. This is the weaker of the two possible capstones only in the sense that it fixes one specific policy; it is chosen over Theorem 8.14 as the goal because it needs the full generality of the workload-based linear Lyapunov argument (Lemma 8.20) with no structural restriction on the network's routing, whereas Theorem 8.14 trades policy generality for a feedforward restriction.

Supporting milestones

Lemma 8.15 isolates the workload derivative identity W˙k(Z(t))=ρk−bk\dot W_k(Z(t)) = \rho_k - b_kW˙k​(Z(t))=ρk​−bk​ at a busy station under any non-idling policy — the calculational engine both Theorem 8.14 and (via Lemma 8.20's analogous linear-potential argument) Theorem 8.18 rely on. Theorem 8.14 shows a feedforward network is stable under any non-idling policy, via a piecewise-linear Lyapunov function built from the routing matrix's block-triangular structure. Lemma 8.20 (numbered in the book but omitted from this mission's planning brief — added here, see STATUS.md) is the direct structural engine behind the goal theorem: a uniform excess departure rate over the total arrival rate at every non-empty class forces extinction. Corollary 8.19 specializes Theorem 8.18 to generalized Jackson networks, where HLSPS provably reduces to non-idling FCFS.

Significance

The result itself. Theorem 8.18 is the book's demonstration that dropping a structural network restriction (feedforward) is possible at the cost of committing to one specific, practically implementable control policy — and Corollary 8.19 shows this specific policy's stability theorem recovers, as a special case, the folklore stability result for the classical multiclass Jackson network under FCFS, arguably the single most studied queueing model in the field. Theorem 8.14, in turn, is the sharpest possible policy-agnostic statement: for feedforward networks, subcriticality alone (with no assumption at all beyond non-idling) suffices.

Formalizing it. Searches for "Jackson network" and "workload" (q=Jackson%20network, q=workload) return no relevant results (per triage.json); this mission is a from-scratch formalization of feedforward networks, the workload operator, HLSPS control, and their stability theorems, building directly on mission III's Theorem 6.2 and mission V's Lyapunov criteria.

Difficulty

Theorem 8.14's proof needs a genuinely delicate construction: a sequence of positive weights δk\delta_kδk​, chosen via the routing matrix's block-triangular structure (guaranteed by feedforwardness) so that the piecewise-linear function H(z)=max⁡kδkWk(z)H(z) = \max_k \delta_k W_k(z)H(z)=maxk​δk​Wk​(z) is positive-definite and has the right drift everywhere — an inductive argument over stations that does not generalize to non-feedforward networks, which is exactly why Theorem 8.18 needs an entirely different (linear, policy-specific) argument rather than a direct strengthening of 8.14's. A second, more subtle difficulty is that Theorem 8.14 and Theorem 8.18 are not related as special case and generalization in the book's own proof structure, despite their overlapping conclusions on feedforward networks under FCFS-like policies: 8.14 is agnostic to policy but needs feedforward structure, while 8.18 is agnostic to structure but needs the specific HLSPS policy — formalizing one as a corollary of the other would misrepresent the book's actual logical dependencies.

Formalization scope

Mission IV's queueing-network model data and fluid-equation specialization are restated locally (drafts in this series do not import one another). The workload operator's matrix inverse (I−P′)−1(I-P')^{-1}(I−P′)−1 is supplied as external data with its defining two-sided-inverse property, rather than derived from substochasticity/transience hypotheses on PPP (Chapter 2 material, out of series scope) — the same convention mission III used for its process-family apparatus. The non-idling and HLSPS fluid models are each packaged as their own predicate plus a Definition-6.3-style stability specialization, and Corollary 8.19 deliberately reuses the non-idling stability object (not a separately restated "FCFS fluid model") since the book's own remark identifies the two exactly for generalized Jackson networks. A formalization that stated Theorem 8.14 as a corollary of Theorem 8.18, or vice versa, would misrepresent the chapter's actual proof architecture (see Difficulty); this mission keeps them as independent milestones/ goal, per BRIEF.md's own instruction. Lemma 8.20, numbered and within this chunk's page range but absent from the planning brief's disposition table, is added as a milestone rather than silently dropped, since it is the structural step the goal theorem's own proof cites by name. QueueingNetworkData, workloadOperator, IsFeedforward, and the non-idling/HLSPS fluid-model predicates are the primary reusable contributions; contributions completing the five by sorry proofs are welcome.

Selected references

  • J. G. Dai and J. Michael Harrison, Processing Networks: Fluid Models and Stability, Cambridge University Press (forthcoming), pre-publication draft 2020-4-2. http://spnbook.org
  • J. G. Dai, "On positive Harris recurrence of multiclass queueing networks: a unified approach via fluid limit models," Annals of Applied Probability 5 (1995), 49–77.
  • M. Bramson, "Convergence to equilibria for fluid models of FIFO queueing networks," Queueing Systems 22 (1996), 5–45.
10 thms2 active usersReviewed
🏆Completed
Algorithmic Game TheoryConvex OptimizationOptimization·Captain: mikedeng1

Existence of an Equilibrium for a Competitive Economy I: Equilibrium Exists When Every Consumer Can Trade Every GoodResearch Paper

Motivation

Walras (1874) described an economy as a system of simultaneous equations, one per market, and argued that a set of prices clearing all markets exists because the number of equations equals the number of unknowns. Counting equations proves nothing, and the question of whether competitive equilibrium exists at all remained open for eighty years. Wald (1935–36) proved existence in special production models under restrictive assumptions on demand. In 1954 Kenneth Arrow and Gérard Debreu gave the first existence proof for a general model with production, many consumers, convex technologies and preferences given by utility indicators (Econometrica 22 (1954) 265–290); McKenzie published an independent proof for a trade model in the same year (Econometrica 22 (1954) 147–161). The resulting Arrow–Debreu model is the reference model of general equilibrium theory and the starting point of market-equilibrium problems in operations research and algorithmic game theory.

The paper proves two existence theorems. This mission is the first, Theorem I, whose key assumption is that every consumer initially holds a positive amount of every commodity. A second mission treats Theorem II, which replaces that assumption by a weaker condition on labour supply.

Setting

There are l≥1l \ge 1l≥1 commodities; a commodity vector is an element of Rl\mathbb R^lRl, compared componentwise (x≧yx \geqq yx≧y means xh≥yhx_h \ge y_hxh​≥yh​ for all hhh; x>yx > yx>y means xh>yhx_h > y_hxh​>yh​ for all hhh). The inner product is p⋅x=∑hphxhp\cdot x = \sum_h p_h x_hp⋅x=∑h​ph​xh​.

  • Producers j=1,…,nj = 1, \dots, nj=1,…,n each have a production set Yj⊆RlY_j \subseteq \mathbb R^lYj​⊆Rl (outputs positive, inputs negative). The aggregate production set is Y=∑jYjY = \sum_j Y_jY=∑j​Yj​.
  • Consumers i=1,…,mi = 1, \dots, mi=1,…,m each have a consumption set Xi⊆RlX_i \subseteq \mathbb R^lXi​⊆Rl, a utility indicator uiu_iui​ on XiX_iXi​, initial holdings ζi∈Rl\zeta_i \in \mathbb R^lζi​∈Rl, and a share αij\alpha_{ij}αij​ of the profit of producer jjj.

Assumptions I–IV are: (I.a) each YjY_jYj​ is closed, convex and contains 000; (I.b) Y∩Ω={0}Y \cap \Omega = \{0\}Y∩Ω={0} with Ω={x≧0}\Omega = \{x \geqq 0\}Ω={x≧0}; (I.c) Y∩(−Y)={0}Y \cap (-Y) = \{0\}Y∩(−Y)={0}; (II) each XiX_iXi​ is closed, convex and bounded from below; (III.a) uiu_iui​ is continuous on XiX_iXi​; (III.b) no xi∈Xix_i \in X_ixi​∈Xi​ is a satiation point; (III.c) if ui(xi)>ui(xi′)u_i(x_i) > u_i(x_i')ui​(xi​)>ui​(xi′​) and 0<t<10 < t < 10<t<1 then ui[txi+(1−t)xi′]>ui(xi′)u_i[t x_i + (1-t) x_i'] > u_i(x_i')ui​[txi​+(1−t)xi′​]>ui​(xi′​); (IV.a) some xi∈Xix_i \in X_ixi​∈Xi​ satisfies xi<ζix_i < \zeta_ixi​<ζi​; (IV.b) αij≥0\alpha_{ij} \ge 0αij​≥0 and ∑iαij=1\sum_i \alpha_{ij} = 1∑i​αij​=1.

A competitive equilibrium is a tuple (x1∗,…,xm∗,y1∗,…,yn∗,p∗)(x_1^*, \dots, x_m^*, y_1^*, \dots, y_n^*, p^*)(x1∗​,…,xm∗​,y1∗​,…,yn∗​,p∗) such that

  1. each yj∗y_j^*yj∗​ maximizes p∗⋅yjp^*\cdot y_jp∗⋅yj​ over YjY_jYj​;
  2. each xi∗x_i^*xi∗​ maximizes uiu_iui​ over {xi∈Xi:p∗⋅xi≦p∗⋅ζi+∑jαij p∗⋅yj∗}\{x_i \in X_i : p^*\cdot x_i \leqq p^*\cdot\zeta_i + \sum_j \alpha_{ij}\, p^*\cdot y_j^*\}{xi​∈Xi​:p∗⋅xi​≦p∗⋅ζi​+∑j​αij​p∗⋅yj∗​};
  3. p∗∈P={p≧0:∑hph=1}p^* \in P = \{p \geqq 0 : \sum_h p_h = 1\}p∗∈P={p≧0:∑h​ph​=1};
  4. z∗=∑ixi∗−∑jyj∗−∑iζiz^* = \sum_i x_i^* - \sum_j y_j^* - \sum_i \zeta_iz∗=∑i​xi∗​−∑j​yj∗​−∑i​ζi​ satisfies z∗≦0z^* \leqq 0z∗≦0 and p∗⋅z∗=0p^*\cdot z^* = 0p∗⋅z∗=0.

An abstract economy (§2) is a game in which each player's feasible set Aι(aˉι)A_\iota(\bar a_\iota)Aι​(aˉι​) depends on the other players' actions aˉι\bar a_\iotaaˉι​; an equilibrium point is a profile at which every player maximizes its pay-off over its feasible set. The proof builds an abstract economy EEE with the consumers, the producers and a fictitious market participant who chooses p∈Pp \in Pp∈P and receives p⋅zp\cdot zp⋅z, and a truncated version E~\tilde EE~ in which all choices are restricted to a large cube CCC.

Formalization targets

Goal: Theorem I

Assumptions I–IV  ⟹  ∃ (x∗,y∗,p∗) satisfying Conditions 1–4.\text{Assumptions I–IV} \implies \exists\, (x^*, y^*, p^*) \text{ satisfying Conditions 1–4.}Assumptions I–IV⟹∃(x∗,y∗,p∗) satisfying Conditions 1–4.

The statement fixes no constants; its only added hypothesis is l≥1l \ge 1l≥1.

Milestones

In the order of the paper's argument:

  1. §1.3.1: under II, III.a and III.c, each uiu_iui​ is quasi-concave on XiX_iXi​.
  2. §1.4.2 (1): Condition 2 together with II, III.b and III.c gives p∗⋅xi∗=p∗⋅ζi+∑jαij p∗⋅yj∗p^*\cdot x_i^* = p^*\cdot\zeta_i + \sum_j \alpha_{ij}\, p^*\cdot y_j^*p∗⋅xi∗​=p∗⋅ζi​+∑j​αij​p∗⋅yj∗​.
  3. Lemma 2.5: an abstract economy with compact convex action sets, continuous pay-offs that are quasi-concave in the player's own action, and continuous constraint correspondences with closed graphs and nonempty convex values has an equilibrium point.
  4. §3.1.2 (2): at an equilibrium point of EEE, Condition 2 holds.
  5. §3.2: every equilibrium point of EEE is a competitive equilibrium.
  6. §3.3.1 (7) and §3.3.2 (2): the attainable sets Y^j\hat Y_jY^j​ and X^i\hat X_iX^i​ (choices compatible with z≦0z \leqq 0z≦0) are bounded.
  7. Remark §3.3.5: if p⋅ζi>min⁡X~ip⋅xip\cdot\zeta_i > \min_{\tilde X_i} p\cdot x_ip⋅ζi​>minX~i​​p⋅xi​, the truncated budget correspondence A~i\tilde A_iA~i​ is continuous at that point.
  8. §3.4.0: for a cube CCC containing every X^i\hat X_iX^i​ and Y^j\hat Y_jY^j​ in its interior, E~\tilde EE~ has an equilibrium point.
  9. §3.4.1: every equilibrium point of E~\tilde EE~ is an equilibrium point of EEE.

Significance

Theorem I shows that the competitive model is consistent: under convexity, continuity, non-satiation and survival assumptions, prices exist at which profit maximization, utility maximization and market clearing hold simultaneously. The welfare theorems, comparative statics, and algorithms that compute equilibria (Scarf's method, market-equilibrium algorithms) presuppose it. Lemma 2.5, Debreu's social-equilibrium theorem (PNAS 38 (1952) 886–893), is also used on its own for generalized Nash equilibrium problems with coupled constraints.

The theorem is proved and textbook material (Debreu, Theory of Value, 1959). To the best of the catalog search, no proof assistant library contains Theorem I, Lemma 2.5, or a Kakutani-type fixed-point theorem for correspondences; on this platform the only related results are Brouwer's fixed-point theorem (AGT.brouwer_fixed_point, proved) and Nash's theorem for finite games (AGT.nash_existence), a special case of Lemma 2.5 with constant constraint sets. The mission produces a machine-checked proof of the paper's argument and, along the way, reusable statements about abstract economies and their equilibria.

Difficulty

The central difficulty is Lemma 2.5. The paper does not prove it but cites Debreu (1952), whose proof uses the Eilenberg–Montgomery fixed-point theorem for correspondences with contractible values. Brouwer's theorem alone does not suffice: the best-response correspondence of an abstract economy is set-valued, and its closed graph must be established from the continuity of the feasible-set correspondences, which is a maximum-theorem argument that Mathlib does not contain. A Kakutani-type fixed-point theorem, or an approximation argument reducing to Brouwer, is needed.

A second difficulty is non-compactness. The economy EEE has unbounded action sets, so the Lemma does not apply to it directly. The boundedness of the attainable sets (§3.3.1) is an asymptotic argument using the irreversibility and no-free-production assumptions I.b and I.c, and the passage from E~\tilde EE~ back to EEE (§3.4.1) needs the attainable choices to lie in the interior of the cube, so that local optimality implies global optimality through III.c. A fixed-point argument on an excess-demand function does not apply directly: demand need not be single-valued or even defined at every price.

Formalization scope

Commodity space is Fin l → ℝ with its componentwise order; the paper's strict vector inequality is written coordinatewise (∀ h, x h < ζ i h), not with the order-theoretic strict inequality on functions. Consumers are Fin m, producers Fin n, and the players of EEE are Fin m ⊕ Fin n ⊕ Unit. Utilities are total functions, and every assumption on them quantifies over XiX_iXi​ only. "Maximizes" is always written as membership plus an inequality against every feasible alternative, never through a supremum. Continuity of a constraint correspondence is the paper's sequential definition (§2.4), a lower-hemicontinuity condition, required at every point of the other players' action space; convergence is asked only in the other players' coordinates.

Two hypotheses are made explicit because the formal statements would otherwise be false: Theorem I and §3.4.0 assume l≥1l \ge 1l≥1 (for l=0l = 0l=0 the price simplex is empty), and Lemma 2.5 assumes every action set nonempty (otherwise, with two or more players and all action sets empty, every hypothesis holds vacuously). The Remark of §3.3.5 carries, as a hypothesis, the nonemptiness of A~i\tilde A_iA~i​ established in §3.3.4. A formalization that makes Theorem I trivial, for instance by taking PPP to contain 000, by reading IV.a with the order-theoretic strict inequality, or by allowing an empty commodity space, is ruled out by these definitions.

A complete development needs: Kakutani's fixed-point theorem (or a Brouwer-based substitute) for compact convex subsets of Rl\mathbb R^lRl; Berge's maximum theorem for the best-response correspondence; the asymptotic-cone argument for §3.3.1; and elementary convex analysis for the budget sets. The abstract-economy definitions and Lemma 2.5 are reusable beyond this mission, including by the Theorem II mission. Contributions of any of these components are welcome, as are alternative proofs of Lemma 2.5.

Selected references

  • K. J. Arrow and G. Debreu, Existence of an Equilibrium for a Competitive Economy, Econometrica 22(3) (1954) 265–290. https://doi.org/10.2307/1907353
  • G. Debreu, A Social Equilibrium Existence Theorem, Proceedings of the National Academy of Sciences 38(10) (1952) 886–893. https://doi.org/10.1073/pnas.38.10.886
  • L. W. McKenzie, On Equilibrium in Graham's Model of World Trade and Other Competitive Systems, Econometrica 22(2) (1954) 147–161. https://doi.org/10.2307/1907539
  • J. Nash, Equilibrium Points in n-Person Games, Proceedings of the National Academy of Sciences 36(1) (1950) 48–49. https://doi.org/10.1073/pnas.36.1.48
  • S. Kakutani, A Generalization of Brouwer's Fixed Point Theorem, Duke Mathematical Journal 8(3) (1941) 457–459. https://doi.org/10.1215/S0012-7094-41-00838-4
  • G. Debreu, Theory of Value: An Axiomatic Analysis of Economic Equilibrium, Wiley, 1959 (Cowles Foundation Monograph 17).
16 thms2 active usersReviewed
🏆Completed
Convex OptimizationOptimizationProbability·Captain: mikedeng1

Dual Stochastic Dominance and Related Mean-Risk Models 1: Second-Degree Stochastic Dominance Is Dominance of Absolute Lorenz CurvesResearch Paper

Motivation

Comparing uncertain outcomes is the basic problem of decision making under risk. Second-degree stochastic dominance (SSD) is the comparison that every risk-averse decision maker who prefers larger outcomes agrees with: XXX dominates YYY in this sense exactly when E U(X)≥E U(Y)\mathbb E\,U(X)\ge\mathbb E\,U(Y)EU(X)≥EU(Y) for every nondecreasing concave utility UUU for which the expectations are finite. The relation grew out of majorization theory for finite distributions (Hardy, Littlewood and Pólya) and was extended to general distributions by Rothschild and Stiglitz and by Hadar and Russell around 1970; it is the standard consistency requirement for portfolio models and for risk measures in operations research and finance.

SSD is defined through the distribution function, which is awkward in optimization: portfolio returns are linear in the decision variables, but their distribution functions are not. Ogryczak and Ruszczyński (SIAM J. Optim. 13 (2002) 60–78) showed that SSD has an equivalent dual description through the integrated quantile function, the absolute Lorenz curve, and that the two descriptions are related by Fenchel conjugation. That dual description underlies the later theory of SSD-constrained optimization (Dentcheva and Ruszczyński, SIAM J. Optim. 14 (2003)) and the use of conditional value-at-risk as an SSD-consistent risk measure.

Setting

Fix a probability space (Ω,B,P)(\Omega,\mathcal B,\mathbb P)(Ω,B,P) and real random variables X,Y:Ω→RX,Y:\Omega\to\mathbb RX,Y:Ω→R with E∣X∣<∞\mathbb E|X|<\inftyE∣X∣<∞, E∣Y∣<∞\mathbb E|Y|<\inftyE∣Y∣<∞.

  • The distribution function is FX(η)=P{X≤η}F_X(\eta)=\mathbb P\{X\le\eta\}FX​(η)=P{X≤η} (Lean: distFun P X).
  • The second performance function is the area below it, FX(2)(η)=∫−∞ηFX(ξ) dξF_X^{(2)}(\eta)=\int_{-\infty}^{\eta}F_X(\xi)\,d\xiFX(2)​(η)=∫−∞η​FX​(ξ)dξ (secondPerformance P X, eq. (2.1)).
  • SSD: X⪰SSDYX\succeq_{SSD}YX⪰SSD​Y iff FX(2)(η)≤FY(2)(η)F_X^{(2)}(\eta)\le F_Y^{(2)}(\eta)FX(2)​(η)≤FY(2)​(η) for every η∈R\eta\in\mathbb Rη∈R (SSD P X Y, eq. (2.2)). The dominating variable has the smaller curve.
  • The first quantile function is the left-continuous inverse FX(−1)(p)=inf⁡{η:FX(η)≥p}F_X^{(-1)}(p)=\inf\{\eta:F_X(\eta)\ge p\}FX(−1)​(p)=inf{η:FX​(η)≥p}, 0<p≤10<p\le10<p≤1 (leftQuantile P X). A number qqq is a ppp-quantile if P{X<q}≤p≤P{X≤q}\mathbb P\{X<q\}\le p\le\mathbb P\{X\le q\}P{X<q}≤p≤P{X≤q} (IsPQuantile P X p q).
  • The second quantile function (absolute Lorenz curve) FX(−2):R→R‾F_X^{(-2)}:\mathbb R\to\overline{\mathbb R}FX(−2)​:R→R is FX(−2)(p)=∫0pFX(−1)(α) dαF_X^{(-2)}(p)=\int_0^pF_X^{(-1)}(\alpha)\,d\alphaFX(−2)​(p)=∫0p​FX(−1)​(α)dα for 0≤p≤10\le p\le10≤p≤1 and +∞+\infty+∞ otherwise (secondQuantile P X, eq. (3.2)).
  • The convex conjugate of F:R→R‾F:\mathbb R\to\overline{\mathbb R}F:R→R is F∗(p)=sup⁡ξ{pξ−F(ξ)}F^*(p)=\sup_\xi\{p\xi-F(\xi)\}F∗(p)=supξ​{pξ−F(ξ)} (conj F), and ∂f(η)\partial f(\eta)∂f(η) is the subdifferential of a real function fff at η\etaη (subdiff f η).

Formalization targets

Goal: Theorem 3.2

X⪰SSDY  ⟺  FX(−2)(p)≥FY(−2)(p)for all 0≤p≤1.X\succeq_{SSD}Y\iff F_X^{(-2)}(p)\ge F_Y^{(-2)}(p)\quad\text{for all }0\le p\le1.X⪰SSD​Y⟺FX(−2)​(p)≥FY(−2)​(p)for all 0≤p≤1.

Both directions are required, and the range of ppp includes both endpoints (at p=1p=1p=1 the right-hand side contains EX≥EY\mathbb EX\ge\mathbb EYEX≥EY).

Milestones, in the order the argument uses them

  1. (2.4): FX(2)(η)=∫−∞η(η−ξ) PX(dξ)=Emax⁡(η−X,0)F_X^{(2)}(\eta)=\int_{-\infty}^{\eta}(\eta-\xi)\,P_X(d\xi)=\mathbb E\max(\eta-X,0)FX(2)​(η)=∫−∞η​(η−ξ)PX​(dξ)=Emax(η−X,0).
  2. §2, p. 62: FX(2)F_X^{(2)}FX(2)​ is continuous, convex, nonnegative and nondecreasing.
  3. §3, p. 64: for p∈(0,1)p\in(0,1)p∈(0,1) the ppp-quantiles form a closed interval with left end FX(−1)(p)F_X^{(-1)}(p)FX(−1)​(p).
  4. (3.3): ∂FX(2)(η)=[P{X<η},P{X≤η}]\partial F_X^{(2)}(\eta)=[\mathbb P\{X<\eta\},\mathbb P\{X\le\eta\}]∂FX(2)​(η)=[P{X<η},P{X≤η}] for every η\etaη.
  5. Theorem 3.1(i): FX(−2)=[FX(2)]∗F_X^{(-2)}=[F_X^{(2)}]^*FX(−2)​=[FX(2)​]∗ on all of R\mathbb RR.
  6. Theorem 3.1(ii): FX(2)=[FX(−2)]∗F_X^{(2)}=[F_X^{(-2)}]^*FX(2)​=[FX(−2)​]∗ on all of R\mathbb RR.

A companion item, Corollary 3.3, states the four equivalent characterizations of a ppp-quantile (quantile condition, attainment in either conjugate, and the Fenchel–Young equality FX(−2)(p)+FX(2)(η)=pηF_X^{(-2)}(p)+F_X^{(2)}(\eta)=p\etaFX(−2)​(p)+FX(2)​(η)=pη).

Significance

Theorem 3.2 converts a condition on distribution functions into a condition on integrated quantiles. Its consequences in the paper include the SSD consistency of the mean–risk models built on tail means (conditional value-at-risk), on the Gini mean difference and on the mean absolute deviation from a quantile, and the linear-programming representations of those models for finitely many scenarios; the companion mission Dual Stochastic Dominance and Related Mean-Risk Models 2 builds on the same objects. Theorem 3.1 is the precise statement that FX(2)F_X^{(2)}FX(2)​ and FX(−2)F_X^{(-2)}FX(−2)​ form a conjugate pair; Corollary 3.3 identifies the subgradients of each with the quantiles of XXX.

All results here are proved in the paper, and the quantile characterization of the increasing concave order also appears in the stochastic-orders literature. None of them is formalized: Mathlib at the pinned revision has ProbabilityTheory.cdf but no convex conjugate on the extended reals, no subdifferential of a real function, no quantile function and no stochastic dominance. The mission produces a machine-checked account of the quantile side of SSD, with the conjugacy stated exactly, including the value +∞+\infty+∞ off [0,1][0,1][0,1].

Difficulty

The naive route to Theorem 3.2 compares FX(2)F_X^{(2)}FX(2)​ and FY(2)F_Y^{(2)}FY(2)​ through the quantile functions directly, but the first quantiles F(−1)F^{(-1)}F(−1) need not be ordered when X⪰SSDYX\succeq_{SSD}YX⪰SSD​Y (the paper notes this on p. 65), so no pointwise argument on quantiles works. The equivalence rests on Theorem 3.1, and there the hard part is computing the conjugate of FX(2)F_X^{(2)}FX(2)​ for a general distribution: atoms of XXX make FX(2)F_X^{(2)}FX(2)​ nondifferentiable and flat pieces of FXF_XFX​ make the maximizer non-unique, so the subdifferential (3.3) and the interval of ppp-quantiles must be handled as sets, and the endpoints p=0,1p=0,1p=0,1 (where the supremum need not be attained) and p∉[0,1]p\notin[0,1]p∈/[0,1] (where it is +∞+\infty+∞) must be treated separately. Part (ii) is a biconjugation statement for a closed convex function, whose general form is not in Mathlib.

Formalization scope

  • One probability space (Ω, P) with [IsProbabilityMeasure P] carries both XXX and YYY; nothing depends on anything but the laws, and no independence is assumed.
  • FX(η)F_X(\eta)FX​(η) is P.real {ω | X ω ≤ η}; FX(2)F_X^{(2)}FX(2)​ is a Bochner integral over Set.Iic η; FX(−2)F_X^{(-2)}FX(−2)​ is an interval integral over (0,p](0,p](0,p], placed in EReal, with ⊤ off [0,1][0,1][0,1].
  • The conjugate is ⨆ ξ, ((p * ξ : ℝ) : EReal) - F ξ in the complete lattice EReal, so terms where F=+∞F=+\inftyF=+∞ contribute −∞-\infty−∞, exactly the paper's convention.
  • Standing assumption. Every item using F(2)F^{(2)}F(2) or F(−2)F^{(-2)}F(−2) assumes Integrable X P (and Integrable Y P in the goal). This is the paper's own hypothesis E∣X∣<∞\mathbb E|X|<\inftyE∣X∣<∞ (p. 65, and the hypothesis of Theorem 3.1), not a repair. The ppp-quantile milestone assumes only AEMeasurable X P.
  • Quantile at p=1p=1p=1. FX(−1)F_X^{(-1)}FX(−1)​ is a real sInf. It is the true infimum for 0<p<10<p<10<p<1; at p=1p=1p=1 the paper's value can be +∞+\infty+∞ while sInf ∅ = 0. This one point does not affect (3.2), and no item states anything about FX(−1)(1)F_X^{(-1)}(1)FX(−1)​(1).
  • Omitted. The conditional-expectation form P{X≤η} E{η−X∣X≤η}\mathbb P\{X\le\eta\}\,\mathbb E\{\eta-X\mid X\le\eta\}P{X≤η}E{η−X∣X≤η} in (2.4) is not stated, since it is undefined when P{X≤η}=0\mathbb P\{X\le\eta\}=0P{X≤η}=0.
  • Trivializing encodings are ruled out. F(2)F^{(2)}F(2) is defined by (2.1), not as Emax⁡(η−X,0)\mathbb E\max(\eta-X,0)Emax(η−X,0), and F(−2)F^{(-2)}F(−2) by (3.2), not as a conjugate; either shortcut would make a milestone or Theorem 3.1 true by definition.
  • Infrastructure and reuse. Welcome contributions: the extended-real conjugate and Fenchel–Young inequality on R\mathbb RR, biconjugation of closed convex functions of one variable, subdifferentials of integrals of monotone functions, and the basic theory of left quantiles (the quantile transform FX(−1)(U)∼XF_X^{(-1)}(U)\sim XFX(−1)​(U)∼X). These are reusable beyond this mission, in particular by mission 2 of this series and by any formalization of conditional value-at-risk. The platform's VectorSpaceOpt.fenchel_biconjugate_on and ConvexOptimization.fenchelConjugate concern real-valued conjugates on other spaces and are related but not reused.

Selected references

  • W. Ogryczak, A. Ruszczyński, Dual stochastic dominance and related mean-risk models, SIAM J. Optim. 13(1) (2002) 60–78. https://doi.org/10.1137/S1052623400375075
  • R. T. Rockafellar, Convex Analysis, Princeton University Press, 1970 (Theorems 12.2 and 23.5 are used in the paper's proofs). https://doi.org/10.1515/9781400873173
  • M. Rothschild, J. E. Stiglitz, Increasing risk: I. A definition, J. Econom. Theory 2 (1970) 225–243. https://doi.org/10.1016/0022-0531(70)90038-4
  • J. Hadar, W. R. Russell, Rules for ordering uncertain prospects, Amer. Econom. Rev. 59 (1969) 25–34. https://www.jstor.org/stable/1811090
  • D. Dentcheva, A. Ruszczyński, Optimization with stochastic dominance constraints, SIAM J. Optim. 14(2) (2003) 548–566. https://doi.org/10.1137/S1052623402420528
  • M. Shaked, J. G. Shanthikumar, Stochastic Orders, Springer, 2007. https://doi.org/10.1007/978-0-387-34675-5
10 thms2 active usersReviewed
🏆Completed
CombinatoricsLinear OptimizationOptimization·Captain: Shuze Chen

Disjunctive Programming XVI: Unions of Upper Monotone Polytopes and PolymatroidsTextbook

Motivation

This mission is the sixteenth and last of the Disjunctive Programming series, and its goal theorem is the book's own closing result. The chapter's arc closes a loop opened at the very start of the book: Theorem 2.1 (02a-convex-hull) gave the convex hull of a union of polyhedra in the same space via lifting; this chapter's Theorem 13.13 (not drafted in this mission — see below) gives the dominant of a union of polytopes in different spaces, and the chapter's final result specializes that machinery to the case where the two polytopes are polymatroids — obtaining a fully explicit, closed-form convex hull in the original variable space, with no lifting at all. Polymatroids are among the most heavily studied objects in combinatorial optimization, from Edmonds's foundational greedy-algorithm characterization onward (J. Edmonds, Submodular functions, matroids, and certain polyhedra, in Combinatorial Structures and Their Applications, Gordon and Breach, 1970, 69–87), and a disjunction of two polymatroids — "satisfy one covering system or the other" — arises naturally whenever two competing combinatorial resource constraints interact.

Setting

Fix a ground set N={1,…,n}N = \{1,\dots,n\}N={1,…,n}. A set function r:2N→Rr : 2^N \to \mathbb{R}r:2N→R is a polymatroid rank function if r(∅)=0r(\emptyset)=0r(∅)=0, rrr is nondecreasing, and rrr is submodular: r(A)+r(B)≥r(A∪B)+r(A∩B)r(A)+r(B) \ge r(A\cup B)+r(A\cap B)r(A)+r(B)≥r(A∪B)+r(A∩B) for all A,B⊆NA,B\subseteq NA,B⊆N. (A related but distinct condition, used earlier in the chapter for "Application 1," additionally requires r(A)≤∣A∣r(A)\le|A|r(A)≤∣A∣ on every proper subset — matroid rank functions satisfy both.) The associated polymatroid is

P(r):={x∈R+n:∑j∈Axj≤r(A) for all A⊆N}.P(r) := \Big\{x \in \mathbb{R}^n_+ : \textstyle\sum_{j\in A} x_j \le r(A) \text{ for all } A \subseteq N\Big\}.P(r):={x∈R+n​:∑j∈A​xj​≤r(A) for all A⊆N}.

For two ground sets M,NM,NM,N and set functions r1,r2r_1,r_2r1​,r2​, the disjoint-space union is Z(r1,r2):={(x,y)∈[0,1]m×[0,1]n:x∈P(r1) or y∈P(r2)}Z(r_1,r_2) := \{(x,y)\in[0,1]^m\times[0,1]^n : x\in P(r_1) \text{ or } y\in P(r_2)\}Z(r1​,r2​):={(x,y)∈[0,1]m×[0,1]n:x∈P(r1​) or y∈P(r2​)}. For polymatroid rank functions r1,r2r_1,r_2r1​,r2​ on the same ground set NNN, Π:={π≥0:πx≤1 for x∈P(r1)∪P(r2)}\Pi := \{\pi \ge 0 : \pi x \le 1 \text{ for } x \in P(r_1)\cup P(r_2)\}Π:={π≥0:πx≤1 for x∈P(r1​)∪P(r2​)} and U:={u≥0:∑AuAri(A)≤1, i=1,2}U := \{u \ge 0 : \sum_A u_A r_i(A) \le 1,\ i=1,2\}U:={u≥0:∑A​uA​ri​(A)≤1, i=1,2} (indexed by all subsets A⊆NA \subseteq NA⊆N) are the auxiliary polytopes the final proof reduces to.

Formalization targets

Proposition 13.16. For set functions r1,r2r_1,r_2r1​,r2​ satisfying the Application-1 conditions,

conv(Z(r1,r2))={(x,y):∣A∣−x(A)∣A∣−r1(A)+∣B∣−y(B)∣B∣−r2(B)≥1 ∀A⊆M,B⊆N with r1(A)<∣A∣, r2(B)<∣B∣}.\mathrm{conv}(Z(r_1,r_2)) = \Big\{(x,y) : \frac{|A|-x(A)}{|A|-r_1(A)} + \frac{|B|-y(B)}{|B|-r_2(B)} \ge 1 \ \forall A\subseteq M, B\subseteq N \text{ with } r_1(A)<|A|,\ r_2(B)<|B|\Big\}.conv(Z(r1​,r2​))={(x,y):∣A∣−r1​(A)∣A∣−x(A)​+∣B∣−r2​(B)∣B∣−y(B)​≥1 ∀A⊆M,B⊆N with r1​(A)<∣A∣, r2​(B)<∣B∣}.

Corollary 13.21. The same-space specialization: conv(P(r1)∪P(r2))={w∈[0,1]n:w=x+y,[the same displayed inequality, A,B⊆N]}\mathrm{conv}(P(r_1)\cup P(r_2)) = \{w\in[0,1]^n : w=x+y, \text{[the same displayed inequality, } A,B\subseteq N\text{]}\}conv(P(r1​)∪P(r2​))={w∈[0,1]n:w=x+y,[the same displayed inequality, A,B⊆N]}.

Proposition 13.22. Π\PiΠ is exactly the projection, onto π\piπ, of {πj≤∑A∋juA (j∈N), ∑AuAri(A)≤1 (i=1,2), π,u≥0}\{\pi_j \le \sum_{A\ni j} u_A\ (j\in N),\ \sum_A u_A r_i(A)\le1\ (i=1,2),\ \pi,u\ge0\}{πj​≤∑A∋j​uA​ (j∈N), ∑A​uA​ri​(A)≤1 (i=1,2), π,u≥0}.

Proposition 13.23. Every extreme point of Π\PiΠ arises from an extreme point of UUU via πj=∑A∋juA\pi_j = \sum_{A\ni j} u_Aπj​=∑A∋j​uA​.

Theorem 13.24 (goal, the book's closing theorem). For polymatroid rank functions r1,r2r_1,r_2r1​,r2​,

conv(P(r1)∪P(r2))={x≥0:x(A)≤max⁡{r1(A),r2(A)} ∀A⊆N;  r2(B)−r1(B)r1(A)r2(B)−r1(B)r2(A)x(A)+r1(A)−r2(A)r1(A)r2(B)−r1(B)r2(A)x(B)≤1\mathrm{conv}(P(r_1)\cup P(r_2)) = \Big\{x\ge0 : x(A)\le\max\{r_1(A),r_2(A)\}\ \forall A\subseteq N;\ \ \frac{r_2(B)-r_1(B)}{r_1(A)r_2(B)-r_1(B)r_2(A)}x(A) + \frac{r_1(A)-r_2(A)}{r_1(A)r_2(B)-r_1(B)r_2(A)}x(B) \le 1conv(P(r1​)∪P(r2​))={x≥0:x(A)≤max{r1​(A),r2​(A)} ∀A⊆N;  r1​(A)r2​(B)−r1​(B)r2​(A)r2​(B)−r1​(B)​x(A)+r1​(A)r2​(B)−r1​(B)r2​(A)r1​(A)−r2​(A)​x(B)≤1  ∀A,B⊆N with (r1(A)−r2(A))(r1(B)−r2(B))<0}.\ \forall A,B\subseteq N \text{ with } (r_1(A)-r_2(A))(r_1(B)-r_2(B))<0\Big\}. ∀A,B⊆N with (r1​(A)−r2​(A))(r1​(B)−r2​(B))<0}.

The targets trace the book's own tower: the disjoint-space specialization (13.16) and its same-space corollary (13.21) establish the lifted description; Propositions 13.22-13.23 build the blocker/projection machinery; Theorem 13.24 collapses everything into the unlifted, original-variable-space closed form that is the book's final word.

Significance

Theorem 13.24 is a genuinely rare achievement in polyhedral combinatorics: a complete, explicit, non-lifted facet description for the union of two polymatroids — objects whose individual facet structure is already exponential and only tractable via the greedy algorithm and submodular minimization. That the union of two such objects still admits a closed form, stated purely in terms of the two rank functions evaluated at pairs of subsets, is the payoff the entire chapter's machinery (dominants, blockers, upper monotonicity, disjoint-space unions) was built toward. The result strictly generalizes an earlier theorem restricted to matroid polyhedra, obtained there by different techniques specific to matroids; this proof works because polymatroid optimization (Edmonds's greedy algorithm) survives in the more general submodular, non-0/1-truncated setting.

Both directions are proved in the source (Balas's own chapter, building on Edmonds's polymatroid theory and the disjoint-union machinery developed earlier in the same chapter) but have no counterpart on this platform: nothing existing treats polymatroids, polymatroid rank functions, or a closed-form union of two polymatroids. Mathlib's Combinatorics/Matroid/* covers matroids and their rank functions but not this strictly more general polymatroid object (an integer- or real-valued submodular monotone set function, not a matroid's 0/1-truncated rank). This mission produces the first Lean statements of all five targets.

Difficulty

The obvious shortcut for Theorem 13.24 is to state only the "single active subset" family of inequalities (x(A)≤max⁡{r1(A),r2(A)}x(A)\le\max\{r_1(A),r_2(A)\}x(A)≤max{r1​(A),r2​(A)}) and treat the two-subset family as a minor addendum — but the two-subset inequalities are not optional refinements, they are half of the facet system, arising from the genuinely two-dimensional case of the underlying linear program (a basic feasible solution of UUU with two nonzero components). Dropping them, or stating them only for a special case of A,BA,BA,B, would produce a strictly weaker (and generally invalid, since it would omit real facets) description.

The condition (r1(A)−r2(A))(r1(B)−r2(B))<0(r_1(A)-r_2(A))(r_1(B)-r_2(B))<0(r1​(A)−r2​(A))(r1​(B)−r2​(B))<0 is easy to state but not to motivate without the underlying linear algebra: it is exactly the condition under which the 2×22\times22×2 system uAr1(A)+uBr1(B)=1u_Ar_1(A)+u_Br_1(B)=1uA​r1​(A)+uB​r1​(B)=1, uAr2(A)+uBr2(B)=1u_Ar_2(A)+u_Br_2(B)=1uA​r2​(A)+uB​r2​(B)=1 has a solution with both uA,uB>0u_A,u_B>0uA​,uB​>0 — a fact the book verifies by direct computation (Cramer's rule) rather than a structural argument, which is why this mission states the condition exactly as derived rather than paraphrasing it into a more "intuitive" but unfaithful form.

Formalization scope

The ambient space is Fin n → ℝ throughout (or Fin m → ℝ / Fin n → ℝ separately for Proposition 13.16's disjoint spaces), matching the series default; subsets A,B⊆NA,B\subseteq NA,B⊆N are Finset (Fin n), and the auxiliary variable uuu of Propositions 13.22-13.23 is indexed by Finset (Fin n) itself (a genuine Fintype for fixed n), matching "uAu_AuA​ for all A⊆NA\subseteq NA⊆N" directly. IsApp1SetFunction and IsPolymatroidRankFunction are kept as two distinct predicates — the goal theorem uses the latter, Proposition 13.16/Corollary 13.21 the former — matching BRIEF.md's explicit warning to locate and preserve the book's own exact numbered conditions rather than infer a single merged notion. A trivializing formalization to rule out explicitly: stating Theorem 13.24 with only the single-subset inequality family, which would omit the two-subset facets that are half of the theorem's actual content.

This mission depends on no other chunk's Lean definitions; it restates 13a-dominants's dominant/blocker/upper-monotone vocabulary only informally (the underlying object, not any specific Lean declaration), per the series convention, since no chunk in this series can import another's draft module. Theorem 13.13 (the general dominant of a disjoint-space union) and Theorem 13.18 (the general same-space reduction) — the two results whose specializations Proposition 13.16 and Corollary 13.21 respectively are — were not drafted this pass; see HARD.md. As the last mission of the whole book, this chunk's items.yaml closes the series begun in 01-intro-duality: sixteen missions, one book, spanning from the founding disjunctive Farkas lemma to this closed-form union of two polymatroids.

Selected references

  • J. Edmonds, Submodular functions, matroids, and certain polyhedra, in Combinatorial Structures and Their Applications, Gordon and Breach, 1970, 69–87 (reprinted in Combinatorial Optimization — Eureka, You Shrink!, LNCS 2570, Springer, 2003, 11–26, https://doi.org/10.1007/3-540-36478-1_2).
  • E. Balas, A. Bockmayr, N. Pisaruk, and L. Wolsey, On unions and dominants of polytopes, Mathematical Programming A 99 (2004), 223–239. https://doi.org/10.1007/s10107-003-0432-4
  • E. Balas, Disjunctive Programming, Springer, 2018, Chapter 13, §13.2.1–13.8 (the book's final chapter). https://doi.org/10.1007/978-3-030-00148-3
6 thms2 active usersReviewed
🏆Completed
Linear OptimizationOptimization·Captain: Shuze Chen

Disjunctive Programming XIV: Disjunctive Cuts from the V-Polyhedral RepresentationTextbook

Motivation

The lift-and-project cut-generating LP (CGLP) is the workhorse of the book's cutting-plane machinery, but its number of variables grows with qqq, the number of terms in the disjunction — a real computational cost for disjunctions with many terms. An alternative representation of the same disjunctive hull, built from vertices and extreme rays rather than from a dual LP, trades this away: its number of variables is fixed at nnn regardless of qqq, at the price of a constraint set that is generally exponential in size (T. H. Kim, V-polyhedral disjunctive cuts, PhD thesis and papers with E. Balas; the underlying representation traces to the classical Minkowski–Weyl theorem for polyhedra). This mission formalizes the chapter's capstone: the V-polyhedral, lift-and-project, and generalized-intersection-cut families — three representations that look structurally different — coincide exactly.

Setting

A disjunctive set in V-polyhedral (vertex-ray) form is F:=⋃h∈QPhF := \bigcup_{h\in Q} P^hF:=⋃h∈Q​Ph, Ph:=conv Vh+cone RhP^h := \mathrm{conv}\,V^h + \mathrm{cone}\,R^hPh:=convVh+coneRh, where VhV^hVh/RhR^hRh are the (finite) sets of vertices and extreme rays of the hhh-th disjunct. The conic hull of a set SSS is the set of all finite nonnegative combinations of its elements. For a reference point xF∈Fx_F \in FxF​∈F, the disjunctive cone CxFC_{x_F}CxF​​ is the homogenization, at xFx_FxF​, of the translated disjunctive system: (x′,x0′)∈Rn×R+(x',x_0') \in \mathbb{R}^n \times \mathbb{R}_+(x′,x0′​)∈Rn×R+​ with Ax′+(AxF−b)x0′≥0Ax' + (Ax_F-b)x_0' \ge 0Ax′+(AxF​−b)x0′​≥0 and ⋁h(Dhx′+(DhxF−d0h)x0′≥0)\bigvee_h(D^hx' + (D^hx_F-d^h_0)x_0' \ge 0)⋁h​(Dhx′+(DhxF​−d0h​)x0′​≥0).

For a relaxation P~h\tilde P^hP~h of each disjunct with Ph⊆P~h⊆C(xh)P^h \subseteq \tilde P^h \subseteq C(x^h)Ph⊆P~h⊆C(xh) (the LP cone at the disjunct's own optimum xhx^hxh), write V~h\tilde V^hV~h, R~h\tilde R^hR~h for its vertices and rays, and C:=conv(⋃hV~h)+cone(⋃hR~h)C := \mathrm{conv}(\bigcup_h \tilde V^h) + \mathrm{cone}(\bigcup_h \tilde R^h)C:=conv(⋃h​V~h)+cone(⋃h​R~h) for the single combined polyhedron they generate. The associated L&P cut-generating LP is

α=uhD~h,β≤uhd~0h(h∈Q),∑h∈Quhe=1,uh≥0.\alpha = u^h \tilde D^h, \qquad \beta \le u^h \tilde d^h_0 \quad (h\in Q), \qquad \textstyle\sum_{h\in Q} u^h e = 1, \qquad u^h \ge 0.α=uhD~h,β≤uhd~0h​(h∈Q),∑h∈Q​uhe=1,uh≥0.

Formalization targets

Proposition 12.1. αx≥β\alpha x \ge \betaαx≥β is valid for FFF if and only if αp≥β\alpha p \ge \betaαp≥β for every p∈Vhp \in V^hp∈Vh and αr≥0\alpha r \ge 0αr≥0 for every r∈Rhr \in R^hr∈Rh, over every h∈Qh \in Qh∈Q.

Proposition 12.3. For a cut αx≥β\alpha x \ge \betaαx≥β tight at xFx_FxF​ (αxF=β\alpha x_F = \betaαxF​=β) and x∈Fx \in Fx∈F: αx<β\alpha x < \betaαx<β if and only if α(x−xF)<0\alpha(x - x_F) < 0α(x−xF​)<0 for the corresponding point (x−xF,1)(x - x_F, 1)(x−xF​,1) of CxFC_{x_F}CxF​​.

Theorem 12.4. If (α,β)(\alpha,\beta)(α,β) satisfies αp≥β\alpha p \ge \betaαp≥β for every p∈V~hp \in \tilde V^hp∈V~h and αr≥0\alpha r \ge 0αr≥0 for every r∈R~hr \in \tilde R^hr∈R~h (over every hhh), and the mixed-integer feasible set PIP_IPI​ lies in the combined polyhedron CCC, then αx≥β\alpha x \ge \betaαx≥β is valid for PIP_IPI​.

Theorem 12.5 (goal). (α,β)(\alpha,\beta)(α,β) is valid for the combined vertex-ray system if and only if there exists a multiplier u={uh}h∈Qu = \{u^h\}_{h\in Q}u={uh}h∈Q​ making it simultaneously a feasible solution of the CGLP above and a generalized intersection cut from

S:={x∈Rn:uhD~hx≤uhd~0h, h∈Q}.S := \{x \in \mathbb{R}^n : u^h \tilde D^h x \le u^h \tilde d^h_0,\ h \in Q\}.S:={x∈Rn:uhD~hx≤uhd~0h​, h∈Q}.

The targets move from the elementary generator-validity fact (12.1) and its algorithmic companion (12.3, which the iterative cut-generation procedure of §12.1 uses to search only adjacent extreme points) through the same validity criterion generalized to a relaxed system (12.4) to the three-way unification (12.5) that is the entire point of introducing the V-polyhedral representation in the first place.

Significance

Theorem 12.5 explains why the V-polyhedral approach is worth having at all: it produces exactly the same cuts as the lift-and-project CGLP, so nothing is lost by switching representations, while the computational cost profile is reversed (the book's own estimate, not part of this mission's targets, shows the V-polyhedral approach at least q3q^3q3 times cheaper for a qqq-term disjunction using P~h=C(xh)\tilde P^h = C(x^h)P~h=C(xh)). This matters directly for disjunctions with many terms — split disjunctions used one or two at a time throughout most of the earlier chapters — which the CGLP approach makes increasingly expensive as qqq grows, but which the V-polyhedral approach handles without a growing variable count.

Both directions are proved in the source (this book's own §12, citing the underlying V-polyhedral cut idea to Balas's joint work with T. H. Kim, and the GIC-to-L&P equivalence to §11.4's own Theorem 11.5) but have no formalized counterpart on this platform: nothing existing treats V-polyhedral representations, disjunctive cones, or a three-way cut-family equivalence. This mission produces the first Lean statements of all four targets.

Difficulty

The obvious shortcut for Theorem 12.5 is to state only "the V-polyhedral cuts and the L&P cuts coincide" and treat the GIC leg as a footnote, since the book's own two-line proof dispatches the GIC equivalence by citing an earlier theorem rather than re-deriving it. But the theorem's actual claim is a three-way equivalence with a specific, described SSS built from the very multipliers that solve the CGLP — dropping the GIC leg, or defining SSS independently of those multipliers, would understate what is being asserted (the book's own remark following the theorem stresses that the GIC-defining points and the V-polyhedral vertices are typically different points that nonetheless yield equivalent cuts, which is exactly the content a two-way statement would erase).

For Theorem 12.4, the subtlety is that CCC (the combined polyhedron) is not the union ⋃hP~h\bigcup_h \tilde P^h⋃h​P~h but its convex hull — a strictly larger set in general — so validity for CCC's generators is a priori a stronger requirement than validity for each P~h\tilde P^hP~h separately; the theorem's force is that this stronger validity is still exactly what is needed (and obtained) to conclude validity for PIP_IPI​.

Formalization scope

The ambient space is Fin n → ℝ throughout, matching the series default, with the disjunction index Q left as a general type for Propositions 12.1/12.3 (so the same Ph/DisjSet definitions serve any finite disjunction) and specialized to [Fintype Q] where a finite sum over disjuncts is needed (Theorem 12.4's combined polyhedron, the CGLP of Theorem 12.5). V^h/R^h (Proposition 12.1) and Ṽ^h/R̃^h (Theorem 12.4) are formalized with the same underlying definitions (Ph, IsVPolyhedralValid) applied to different vertex/ray data, per BRIEF.md's explicit warning that these are distinct objects — not by duplicating the definitions under two names. "Is a generalized intersection cut from SSS" (IsGICFromS) is formalized via the exact characterization Theorem 11.4's own remark in 11a-intersection-cuts gives for the GIC family (valid outside SSS's interior, and a genuine cut), rather than by re-deriving the underlying extreme-ray construction — a trivializing formalization this mission rules out would instead drop this leg's dependence on the same multiplier u that witnesses the CGLP leg, decoupling S from the solution it is supposed to come from.

This mission depends on no other chunk's Lean definitions; it restates 02a-convex-hull's vertex/extreme-point vocabulary, 11a-intersection-cuts's cut apparatus, and 11b-monoidal-strengthening's disjunctive-cut conventions only informally, per the series convention. Theorem 12.2 (the extreme-ray/edge correspondence underlying the "adjacent vertices only" search strategy) was not drafted this pass — see HARD.md — since a faithful, non-circular formalization of "edge of a polytope incident with a point" needs face-lattice machinery beyond what any earlier chunk in this series has built. The ConicHull/DisjunctiveCone/CombinedC definitions are reusable by any later mission touching V-polyhedral cut generation.

Selected references

  • E. Balas and T. H. Kim, Cutting planes from extended LP formulations, Mathematical Programming 156 (2016), 587–606. https://doi.org/10.1007/s10107-015-0885-2
  • E. Balas and M. Perregaard, Generalized intersection cuts and a new cut generating paradigm, Mathematical Programming A 137 (2013), 19–35. https://doi.org/10.1007/s10107-011-0483-x
  • A. Kazachkov, Non-Recursive Cut Generation, PhD dissertation, Carnegie Mellon University, 2018 (cited by Balas for the relaxation-based V-polyhedral cut generator of §12.2).
  • E. Balas, Disjunctive Programming, Springer, 2018, Chapter 12. https://doi.org/10.1007/978-3-030-00148-3
5 thms2 active usersReviewed
🏆Completed
Linear OptimizationOptimization·Captain: Shuze Chen

Disjunctive Programming XII: Intersection Cuts, Generalized Intersection Cuts, and Lift-and-Project CutsTextbook

Motivation

Intersection cuts (Balas, 1971) are the founding construction of cutting-plane theory for mixed 0-1 and mixed-integer programs: given a fractional LP solution xˉ\bar xxˉ and a convex region SSS around it known to contain no feasible integer point, the hyperplane through the points where SSS's boundary meets the extreme rays of the LP cone at xˉ\bar xxˉ cuts off xˉ\bar xxˉ without cutting off any feasible solution. What makes intersection cuts foundational rather than merely one technique among many is a completeness question: do intersection cuts, iterated over every choice of cutting region, exhaust the strongest possible cuts — the facets of the integer hull itself — or only some weaker subclass? Balas answered this affirmatively for standard intersection cuts (those derived from convex sets free of feasible integer points, as originally defined), while a narrower, more recently popular variant restricted to lattice-free sets provably falls short of this completeness (E. Balas, Intersection Cuts — A New Type of Cutting Planes for Integer Programming, Operations Research 19 (1971), 19–39, https://doi.org/10.1287/opre.19.1.19). This mission formalizes that completeness theorem, together with a companion pair of results (Balas and Kis, 2016) pinning down exactly when a lift-and-project cut — a strictly more general cutting-plane construction from an arbitrary disjunction — coincides with a standard intersection cut, and what happens when it provably does not (E. Balas and T. Kis, On the relationship between standard intersection cuts, lift-and-project cuts, and generalized intersection cuts, Mathematical Programming A 160 (2016), 85–114, https://doi.org/10.1007/s10107-015-0975-1).

Setting

Fix a finite index set ι\iotaι (structural and surplus variables of an LP relaxation together), a basic index set I⊆ιI \subseteq \iotaI⊆ι and a nonbasic (cobasis) set JJJ, with optimal simplex-tableau coefficients aˉij\bar a_{ij}aˉij​ for i∈Ii \in Ii∈I, j∈Jj \in Jj∈J. The extreme ray of the LP cone C(J)C(J)C(J) at a basic solution xˉ\bar xxˉ associated with j∈Jj \in Jj∈J has direction rjr^jrj with rij=−aˉijr^j_i = -\bar a_{ij}rij​=−aˉij​ for i∈Ii \in Ii∈I, rjj=1r^j_j = 1rjj​=1, and rij=0r^j_i = 0rij​=0 otherwise; C(J)C(J)C(J) itself is the cone with apex xˉ\bar xxˉ generated by these n=∣J∣n = |J|n=∣J∣ rays. A convex set SSS is PIP_IPI​-free at xˉ\bar xxˉ if xˉ\bar xxˉ lies in int S\mathrm{int}\, SintS and int S\mathrm{int}\, SintS contains no point of the mixed-integer feasible set PIP_IPI​. The standard intersection cut (SIC) derived from such an SSS is ∑j∈J1λjxj≥1\sum_{j\in J} \tfrac{1}{\lambda_j} x_j \ge 1∑j∈J​λj​1​xj​≥1, where λj\lambda_jλj​ is the largest t≥0t \ge 0t≥0 with xˉ−trj∈S\bar x - t r^j \in Sxˉ−trj∈S.

The corner polyhedron corner(J)\mathrm{corner}(J)corner(J) is the convex hull of the integer points contained in C(J)C(J)C(J); it satisfies C(J)⊃corner(J)⊃conv(PI)C(J) \supset \mathrm{corner}(J) \supset \mathrm{conv}(P_I)C(J)⊃corner(J)⊃conv(PI​). A set FFF is a facet of a polyhedron QQQ if it is a proper extreme subset of QQQ of affine dimension exactly dim⁡(Q)−1\dim(Q) - 1dim(Q)−1.

For a lift-and-project cut, fix P:={x:A~x≥b~}P := \{x : \tilde A x \ge \tilde b\}P:={x:A~x≥b~} and a family of inequalities dtx≥d0td^t x \ge d^t_0dtx≥d0t​, t∈Tt \in Tt∈T, presenting a PIP_IPI​-free polyhedron S:={x:dtx≤d0t, t∈T}S := \{x : d^t x \le d^t_0,\ t\in T\}S:={x:dtx≤d0t​, t∈T}. The associated cut-generating LP (CGLP) constraint set (11.6) is

α−utA~−u0tdt=0,−β+utb~+u0td0t=0 (t∈T),∑t∈T(ute+u0t)=1,ut,u0t≥0,\alpha - u^t \tilde A - u^t_0 d^t = 0, \qquad -\beta + u^t \tilde b + u^t_0 d^t_0 = 0 \ (t\in T), \qquad \textstyle\sum_{t\in T}(u^t e + u^t_0) = 1, \qquad u^t, u^t_0 \ge 0,α−utA~−u0t​dt=0,−β+utb~+u0t​d0t​=0 (t∈T),∑t∈T​(ute+u0t​)=1,ut,u0t​≥0,

whose feasible solutions (α,β,{ut,u0t})(\alpha,\beta,\{u^t,u^t_0\})(α,β,{ut,u0t​}) correspond to valid lift-and-project (L&P) cuts αx≥β\alpha x \ge \betaαx≥β for the disjunction built from PPP and the terms dtx≥d0td^t x \ge d^t_0dtx≥d0t​. An inequality γ1x≥γ01\gamma^1 x \ge \gamma^1_0γ1x≥γ01​ dominates γ2x≥γ02\gamma^2 x \ge \gamma^2_0γ2x≥γ02​ on PPP if every x∈Px \in Px∈P satisfying the first also satisfies the second.

Formalization targets

Theorem 11.2 (goal). Every facet FFF of conv(PI)\mathrm{conv}(P_I)conv(PI​), defined by φx≥φ0\varphi x \ge \varphi_0φx≥φ0​ and cutting off some vertex vvv of PPP (i.e. φv<φ0\varphi v < \varphi_0φv<φ0​), is realized exactly by the standard intersection cut derived at vvv from T:={x:φx≤φ0}T := \{x : \varphi x \le \varphi_0\}T:={x:φx≤φ0​}:

T is PI-free at v,{x:1≤∑j∈J1λjxj}={x:φ0≤φx}.T \text{ is } P_I\text{-free at } v, \qquad \Big\{x : 1 \le \textstyle\sum_{j\in J} \tfrac{1}{\lambda_j} x_j\Big\} = \{x : \varphi_0 \le \varphi x\}.T is PI​-free at v,{x:1≤∑j∈J​λj​1​xj​}={x:φ0​≤φx}.

Corollary 11.3. Every vertex of a corner polyhedron not already in conv(PI)\mathrm{conv}(P_I)conv(PI​) is cut off by some standard intersection cut — the same completeness claim restated at the level of individual excluded vertices rather than facets.

Theorem 11.9. A sufficient condition for an L&P cut to reduce to a standard intersection cut: if a basic feasible CGLP solution's multipliers utu^tut are all supported on a single common nonsingular cobasis ι\iotaι, then

{x:β≤αx}={x:1≤∑jπj sj(x)}\{x : \beta \le \alpha x\} = \{x : 1 \le \textstyle\sum_j \pi_j\, s_j(x)\}{x:β≤αx}={x:1≤∑j​πj​sj​(x)}

for the intersection cut with coefficients πj:=max⁡tπjt\pi_j := \max_{t} \pi^t_jπj​:=maxt​πjt​, πjt:=dt(−aˉj)/(d0t−dtaˉ0)\pi^t_j := d^t(-\bar a_j)/(d^t_0 - d^t \bar a_0)πjt​:=dt(−aˉj​)/(d0t​−dtaˉ0​), expressed via the surplus values sjs_jsj​ at ι\iotaι's rows.

Theorem 11.11. When Theorem 11.9's condition fails — even after every positive rescaling of the solution — no intersection cut from SSS is equivalent to the L&P cut; and when the solution additionally uniquely minimizes the CGLP objective, the L&P cut is strictly better than, and dominated by none of, every intersection cut from SSS.

The targets move from the completeness statement itself (11.2, its vertex-level restatement 11.3) to the mechanism explaining why completeness holds in general: a sufficient condition for literal coincidence (11.9), and a proof that failure of that condition is never fatal to completeness because the L&P cut remains at least as strong, in a precise domination sense (11.11).

Significance

Theorem 11.2 is the theoretical justification for standard intersection cuts as a complete cutting plane paradigm: no facet of the integer hull is out of reach of some choice of PIP_IPI​-free cutting region, in sharp contrast to the restricted (lattice-free) variant that dominates the modern multi-row cut literature but is provably incomplete in this sense. Theorems 11.9 and 11.11 locate lift-and-project cuts precisely relative to this complete family: L&P cuts specialize exactly to intersection cuts under an explicit, checkable structural condition on the CGLP solution, and strictly dominate the intersection-cut family whenever that condition cannot be met — which is what makes lift-and-project the strictly more general (and, on general non-split disjunctions, strictly more powerful) construction.

Both directions are proved in the source (Balas 1971 for Theorem 11.2; Balas and Kis 2016 for Theorems 11.9 and 11.11) but have no counterpart on this platform: nothing existing treats intersection cuts, corner polyhedra, cut-generating LPs, or the correspondence between these two cutting-plane families. This mission produces the first Lean statements of all four.

Difficulty

The obvious shortcut for Theorem 11.2 is to treat "cuts off a vertex" and "is PIP_IPI​-free" as producing merely some valid cut, and stop there — Theorem 1.1 already guarantees that much. The actual content is the equality: the specific intersection cut constructed from the halfspace TTT does not just happen to be valid, it reconstructs φ\varphiφ itself, coefficient for coefficient, because TTT is a single hyperplane so every one of the LP cone's nnn extreme rays exits it through the same boundary. Losing sight of this collapses the theorem into a restatement of Theorem 1.1 with no new content.

For Theorem 11.11, the difficulty is that "no intersection cut from SSS is equivalent" must survive scaling: a naive argument might rule out one specific (α,β)(\alpha,\beta)(α,β)-representative satisfying Theorem 11.9's condition while missing that a positive rescaling of the same cut could still satisfy it under a different multiplier vector. The theorem's hypothesis is deliberately built to close this gap by quantifying over every positive scalar and every feasible solution realizing the rescaled pair, not just the given one.

Formalization scope

The ambient space is a generic finite index type ι for Theorem 11.2/Corollary 11.3 (structural and surplus variables together, matching 01-intro-duality's own convention for the intersection- cut apparatus), and Fin n → ℝ for the CGLP-based Theorems 11.9/11.11, matching the series' default. P_I is left as an abstract parameter throughout (never expanded into an explicit integrality predicate on a specific coordinate subset for Theorem 11.2, matching 01-intro- duality's own treatment), except in the corner-polyhedron definitions, where it is made concrete via a coordinate set Nprime since the corollary's statement depends on it directly. "Facet" and "extreme ray" are restated from 02b-polarity's conventions (affine dimension via Module.finrank of vectorSpan; IsExtreme) rather than reinvented, since Chapter 2 already pins these down precisely for this series. "Basic feasible solution" to the CGLP in Theorem 11.9 is captured entirely by the theorem's own submatrix-support condition, not through a separate, independently-derived basicness predicate — the book's own proof uses no other property of basicness, so adding one would be unused decoration, not additional fidelity. Cut equivalence throughout is formalized as exact set equality of the two halfspaces, matching the series' established convention (e.g. 10-split-closure's Theorem 10.1) for what "equivalent cuts" means.

This mission depends on no other chunk's Lean definitions: the intersection-cut apparatus (extremeRay, PIFree) is restated from 01-intro-duality, the facet apparatus (PolyDim, IsFacet) from 02b-polarity, and the tableau apparatus (Ahat, Bhat, Abar, Abar0, SurplusM) from 08-cut-correspondence/09-simplex-tableau/10-split-closure, per the series convention against importing another draft mission's definitions while chunks are drafted concurrently. A trivializing formalization to rule out explicitly: collapsing Theorem 11.2 to "some intersection cut is valid and cuts off vvv" (already implied by Theorem 1.1 alone) rather than the literal set-equality with the facet's own inequality, which is this theorem's actual content.

Selected references

  • E. Balas, Intersection Cuts — A New Type of Cutting Planes for Integer Programming, Operations Research 19 (1971), 19–39. https://doi.org/10.1287/opre.19.1.19
  • E. Balas and T. Kis, On the relationship between standard intersection cuts, lift-and-project cuts, and generalized intersection cuts, Mathematical Programming A 160 (2016), 85–114. https://doi.org/10.1007/s10107-015-0975-1
  • E. Balas and M. Perregaard, Generalized intersection cuts and a new cut generating paradigm, Mathematical Programming A 137 (2013), 19–35. https://doi.org/10.1007/s10107-011-0483-x
  • E. Balas, Disjunctive Programming, Springer, 2018, Chapter 11, §11.1–11.5. https://doi.org/10.1007/978-3-030-00148-3
7 thms2 active usersReviewed
🏆Completed
Linear OptimizationOptimization·Captain: Shuze Chen

Disjunctive Programming XI: The Cut-Generating LP Under a Ray NormalizationTextbook

Motivation

Lift-and-project (L&P) cuts strengthen the linear relaxation of a mixed 0-1 program by separating a fractional point from the convex hull of a disjunction such as xk≤0∨xk≥1x_k \le 0 \lor x_k \ge 1xk​≤0∨xk​≥1. Generating an optimal L&P cut means solving the cut-generating linear program (CGLP), a linear program lifted to a space with one new pair of variables per constraint of the original tableau — considerably larger than the tableau itself. Balas and Bonami showed that this higher-dimensional LP need not be solved explicitly at all: an optimal (or near-optimal) L&P cut can instead be produced by ordinary simplex pivots in the original LP tableau, each such pivot implicitly performing an entire block of pivots in the CGLP (E. Balas and P. Bonami, Generating lift-and-project cuts from the LP simplex tableau: open source implementation and testing of new variants, Mathematical Programming Computation 1 (2009), 165–199, https://doi.org/10.1007/s12532-009-0006-4). This correspondence is what made L&P cuts practical in commercial solvers: Perregaard's implementation in XPRESS needed only 5% of the iterations and 1.5% of the time of solving the CGLP explicitly, and Bonami's public implementation in COIN-OR put the method within reach of any solver.

A second, independent line of work asks how the CGLP's feasible region should be normalized. The textbook normalization (fixing the sum of the CGLP multipliers to 111) is scale-dependent — rescaling one constraint of the original system changes which cut the CGLP returns — so Balas and Perregaard proposed the ray normalization αy=1\alpha y = 1αy=1 instead (E. Balas and M. Perregaard, Lift-and-project for mixed 0-1 programming: recent progress, Discrete Applied Mathematics 123 (2002), 129–154, https://doi.org/10.1016/S0166-218X(01)00340-7). Under this normalization the CGLP's optimal value has a clean geometric meaning: it is exactly the distance, measured along a fixed ray from the point being separated, to the convex hull of the disjunctive set. This mission formalizes both results: the pivot correspondence (Theorem 10.1) and the optimal-value characterization under the ray normalization (Theorem 10.2, Theorem 10.3, and Corollary 10.4).

Setting

Fix a finite index set MMM for the rows of a simplex tableau over nnn variables, a matrix A∈RM×nA \in \mathbb R^{M \times n}A∈RM×n, and a right-hand side b:M→Rb : M \to \mathbb Rb:M→R, so that the tableau reads Ax≥bA x \ge bAx≥b (a "tilde" is dropped from the informal A~,b~\tilde A, \tilde bA~,b~ notation for the optimal-basis tableau of the linear relaxation). A basis is an injection ι:Fin n→M\iota : \mathrm{Fin}\, n \to Mι:Finn→M picking out nnn of the rows; write A^\hat AA^ for the n×nn \times nn×n submatrix A^ij=Aι(i),j\hat A_{ij} = A_{\iota(i), j}A^ij​=Aι(i),j​ and b^\hat bb^ for the corresponding subvector. From these, the standard tableau quantities are read off: aˉk0:=ekA^−1b^\bar a_{k0} := e_k \hat A^{-1} \hat baˉk0​:=ek​A^−1b^, aˉkj:=−(A^−1)kj\bar a_{kj} := -(\hat A^{-1})_{kj}aˉkj​:=−(A^−1)kj​, and the surplus of row i∈Mi \in Mi∈M at a point xxx, Surplusi(x):=(Ax−b)i\mathrm{Surplus}_i(x) := (Ax - b)_iSurplusi​(x):=(Ax−b)i​.

Fix a distinguished row kkk with a fractional basic variable, and a candidate pivot row i≠ki \ne ki=k. For ℓ\ellℓ ranging over the nonbasic columns JJJ, set γℓ:=−aˉkℓ/aˉiℓ\gamma_\ell := -\bar a_{k\ell}/\bar a_{i\ell}γℓ​:=−aˉkℓ​/aˉiℓ​; this is the value of a parameter γ\gammaγ at which the combined source row

xk+γxi+∑j∈J(aˉkj+γaˉij)xj=aˉk0+γaˉi0(10.1γ)x_k + \gamma x_i + \sum_{j \in J} (\bar a_{kj} + \gamma \bar a_{ij}) x_j = \bar a_{k0} + \gamma \bar a_{i0} \tag{10.1$_\gamma$}xk​+γxi​+j∈J∑​(aˉkj​+γaˉij​)xj​=aˉk0​+γaˉi0​(10.1γ​)

has its jjj-th coefficient pass through 000. The simple disjunctive cut obtained by applying the split disjunction z≤0∨z≥1z \le 0 \lor z \ge 1z≤0∨z≥1 (where zzz is the left side of (10.1γ_\gammaγ​)) to this row is the object CombinedCutSet.

On the CGLP side, (CGLP)k(\mathrm{CGLP})_k(CGLP)k​ is the cut-generating LP associated with the disjunction −xk≥0∨xk≥1-x_k \ge 0 \lor x_k \ge 1−xk​≥0∨xk​≥1 from Chapter 8: it has one pair of nonnegative multiplier variables (uρ,vρ)(u_\rho, v_\rho)(uρ​,vρ​) per row ρ∈M\rho \in Mρ∈M, plus u0,v0≥0u_0, v_0 \ge 0u0​,v0​≥0, tied together by the normalization ∑ρuρ+u0+∑ρvρ+v0=1\sum_\rho u_\rho + u_0 + \sum_\rho v_\rho + v_0 = 1∑ρ​uρ​+u0​+∑ρ​vρ​+v0​=1, and its feasible solutions (α,u,u0,v,v0,β)(\alpha, u, u_0, v, v_0, \beta)(α,u,u0​,v,v0​,β) correspond exactly to valid cuts αx≥β\alpha x \ge \betaαx≥β for the disjunction. A basic feasible solution to (CGLP)k(\mathrm{CGLP})_k(CGLP)k​ is described by a valid partition (M1,M2)(M_1, M_2)(M1​,M2​) of the nonbasic rows, with uρ=0u_\rho = 0uρ​=0 off M1M_1M1​ and vρ=0v_\rho = 0vρ​=0 off M2M_2M2​.

Separately, fix a disjunctive set and write PD⊆RnP_D \subseteq \mathbb R^nPD​⊆Rn for its convex hull — the object every cut ultimately wants to separate a point from. For a fixed direction y∈Rny \in \mathbb R^ny∈Rn and point xˉ∈Rn\bar x \in \mathbb R^nxˉ∈Rn, (CGLP)y(\mathrm{CGLP})_y(CGLP)y​ is the cut-generating LP under the ray normalization: pairs (α,β)(\alpha, \beta)(α,β) with αx≥β\alpha x \ge \betaαx≥β valid for every x∈PDx \in P_Dx∈PD​ and αy=1\alpha y = 1αy=1, minimizing the objective αxˉ−β\alpha \bar x - \betaαxˉ−β.

Formalization targets

Theorem 10.1. For a genuine ordered pivot chain j1,…,jtj_1, \dots, j_tj1​,…,jt​ inside JJJ (no repeats, each consecutive pair flipping the sign of aˉk,⋅\bar a_{k,\cdot}aˉk,⋅​ as γ\gammaγ increases — rule (b) of the theorem), the simple disjunctive cut from the combined row at γ=γjt\gamma = \gamma_{j_t}γ=γjt​​ equals the lift-and-project cut {x:β≤αx}\{x : \beta \le \alpha x\}{x:β≤αx} associated with a basic feasible solution to (CGLP)k(\mathrm{CGLP})_k(CGLP)k​ for the resulting basis J′:=(J∪{i})∖{jt}J' := (J \cup \{i\}) \setminus \{j_t\}J′:=(J∪{i})∖{jt​}:

CombinedCutSet(k,i,J,γjt)={x:β≤αx}.\mathrm{CombinedCutSet}(k, i, J, \gamma_{j_t}) = \{x : \beta \le \alpha x\}.CombinedCutSet(k,i,J,γjt​​)={x:β≤αx}.

Theorem 10.2. If (CGLP)y(\mathrm{CGLP})_y(CGLP)y​ is feasible, it has a finite minimum if and only if the ray meets the disjunctive hull:

finite min  ⟺  ∃ λ∈R, xˉ+λy∈PD.\text{finite min} \iff \exists\, \lambda \in \mathbb R,\ \bar x + \lambda y \in P_D.finite min⟺∃λ∈R, xˉ+λy∈PD​.

Theorem 10.3 (goal). If (CGLP)y(\mathrm{CGLP})_y(CGLP)y​ has an optimal solution (α~,β~)(\tilde\alpha, \tilde\beta)(α~,β~​), its optimal value is exactly the signed distance to PDP_DPD​ along the ray, and the corresponding boundary point lies exactly on the optimal hyperplane:

xˉTα~−β~=λ∗:=min⁡{λ:xˉ+λy∈PD},(xˉ+λ∗y)Tα~=β~.\bar x^{\mathsf T} \tilde\alpha - \tilde\beta = \lambda^* := \min\{\lambda : \bar x + \lambda y \in P_D\}, \qquad (\bar x + \lambda^* y)^{\mathsf T} \tilde\alpha = \tilde\beta.xˉTα~−β~​=λ∗:=min{λ:xˉ+λy∈PD​},(xˉ+λ∗y)Tα~=β~​.

Corollary 10.4. Taking y:=x∗−xˉy := x^* - \bar xy:=x∗−xˉ for a point x∗x^*x∗ in the lifted polyhedron PQP_QPQ​ gives an optimal solution whose hyperplane separates xˉ\bar xxˉ and meets the segment (xˉ,x∗](\bar x, x^*](xˉ,x∗] at the point closest to x∗x^*x∗.

The targets are ordered from the purely combinatorial pivot correspondence (10.1, independent of the ray normalization) through the abstract feasibility/boundedness dichotomy (10.2) to the concrete value formula that is this mission's goal (10.3), with the geometric illustration (10.4) as a companion result using the same machinery with a specific choice of ray.

Significance

Theorem 10.1 is the theoretical justification for every commercial L&P-cut implementation cited above: it says the pivot correspondence is not an approximation or a heuristic shortcut but an exact identity between a single LP pivot and a specific, describable sequence of CGLP pivots, which is what lets a solver generate an (quasi-)optimal L&P cut at the cost of ordinary simplex pivots instead of solving a much larger LP. Theorem 10.3 gives the ray-normalized CGLP an exact geometric meaning — its value is a distance, not merely a linear-programming optimum — which is what makes the ray normalization the more robust alternative to the scale-dependent constant-sum normalization used elsewhere in the book (§9), and is the basis for the geometric picture (Corollary 10.4, Fig. 10.3) of how a lift-and-project cut relates to the lifted polyhedron PQP_QPQ​.

Both directions are proved in the source text (Balas and Bonami 2009 for Theorem 10.1; Balas and Perregaard 2002 for Theorems 10.2/10.3 and Corollary 10.4) but have no formalized counterpart on this platform: no existing item treats cut-generating LPs, ray normalizations of a projection cone, or the correspondence between two different pivoting processes. This mission produces the first Lean statements of both.

Difficulty

The obvious temptation for Theorem 10.1 is to existentially weaken "the sequence of ttt pivots defined as follows" to "there exists some sequence of pivots realizing the same cut" — which would be true but not what the theorem says, and would erase the entire content that makes the result useful (an algorithm, not just an existence claim). The formalization instead carries the explicit ordered chain j1 :: middle ++ [jt] as data, with the three-part construction (rules (a), (b), (c)) encoded as hypotheses on that specific list via List.IsChain, so the theorem proved is the constructive one the book states, not a weaker existential shadow of it.

For Theorem 10.3, the proof pattern in the book resists a shortcut: showing λ0=λ∗\lambda_0 = \lambda^*λ0​=λ∗ requires deriving a contradiction from each strict inequality (λ0>λ∗\lambda_0 > \lambda^*λ0​>λ∗ violates optimality of the point on PDP_DPD​'s boundary; λ0<λ∗\lambda_0 < \lambda^*λ0​<λ∗ contradicts optimality of (α~,β~)(\tilde\alpha, \tilde\beta)(α~,β~​) for (CGLP)y(\mathrm{CGLP})_y(CGLP)y​ via a competing separating hyperplane), so there is no way to avoid formalizing both directions of the boundedness dichotomy already needed for Theorem 10.2 first.

Formalization scope

The ambient space is Fin n→R\mathrm{Fin}\ n \to \mathbb RFin n→R throughout, matching the rest of the series. (CGLP)y(\mathrm{CGLP})_y(CGLP)y​'s feasibility (IsCGLPYFeasible) is stated directly as validity of (α,β)(\alpha, \beta)(α,β) for PDP_DPD​ under αy=1\alpha y = 1αy=1, not through an explicit representation of the projection cone's extreme rays — this matches how the book's own Theorems 10.2/10.3 and Corollary 10.4 are phrased purely in terms of (α,β)(\alpha,\beta)(α,β)-validity for PDP_DPD​, never in terms of a specific disjunction's multipliers, so this is not a weakening relative to the source. PDP_DPD​ (the disjunctive hull that (CGLP)y(\mathrm{CGLP})_y(CGLP)y​ is defined against) and PQP_QPQ​ (the lifted polyhedron whose supporting hyperplane Corollary 10.4 describes) are kept as two independent Set (Fin n → ℝ) parameters with no assumed relationship between them, matching the book's own text, which never states one; conflating them would be a trivializing formalization that this mission explicitly avoids. "The point closest to x∗x^*x∗" on the segment (xˉ,x∗](\bar x, x^*](xˉ,x∗] is formalized via IsGreatest on the parameter t∈(0,1]t \in (0, 1]t∈(0,1] at which the optimal hyperplane meets the segment, rather than via an unformalized Euclidean-distance minimization, since that is what "closest" means for points colinear with xˉ\bar xxˉ and x∗x^*x∗ on a single ray.

Corollary 10.4 corrects a typo in the printed text: the corollary as printed reads "let y:=xˉy := \bar xy:=xˉ for some x∗∈PQx^* \in P_Qx∗∈PQ​", omitting "x∗−x^* -x∗−" before xˉ\bar xxˉ; the very next line's figure caption gives the intended formula unambiguously as y=x∗−xˉy = x^* - \bar xy=x∗−xˉ, and the formalization uses the corrected formula (see MODERATION_NOTES.md).

This mission depends on no other chunk's Lean definitions — the CGLP and tableau apparatus needed here (originally introduced in Chapters 8 and 9) is restated locally, per the series' convention against importing another draft mission's definitions across chunks that are being drafted concurrently. A complete development needs: Farkas-type separation for the boundedness dichotomy in Theorem 10.2, and careful bookkeeping of finite index sets and their images under the basis maps ι,ι′\iota, \iota'ι,ι′ for Theorem 10.1. The tableau infrastructure (Ahat, Bhat, Abar0, Abar, GammaOf) is reusable by any later mission touching the simplex-tableau side of lift-and-project cuts.

Selected references

  • E. Balas and P. Bonami, Generating lift-and-project cuts from the LP simplex tableau: open source implementation and testing of new variants, Mathematical Programming Computation 1 (2009), 165–199. https://doi.org/10.1007/s12532-009-0006-4
  • E. Balas and M. Perregaard, Lift-and-project for mixed 0-1 programming: recent progress, Discrete Applied Mathematics 123 (2002), 129–154. https://doi.org/10.1016/S0166-218X(01)00340-7
  • E. Balas, Disjunctive Programming, Springer, 2018, Chapter 10, §10.1 and §10.6. https://doi.org/10.1007/978-3-030-00148-3
7 thms2 active usersReviewed
🏆Completed
Linear OptimizationOptimization·Captain: Shuze Chen

Disjunctive Programming X: Solving the Cut-Generating LP on the Simplex TableauTextbook

Motivation

Chapter 8 established an exact correspondence between lift-and-project cuts and simple disjunctive cuts, but its practical payoff is what this chapter develops: the cut-generating LP (CGLP)_k never needs to be formulated or solved on its own. Every pivot of (CGLP)_k can instead be mimicked directly on the much smaller simplex tableau of the original LP relaxation — replacing a large auxiliary linear program with bookkeeping on a tableau the solver already has. This chapter works out that correspondence at the level of individual pivots: which tableau pivot improves the resulting cut, and by how much, answered entirely in terms of ordinary tableau coefficients and two closed-form evaluation functions.

Setting

S:={1,…,m+p} and N:={m+p+1,…,m+p+n} index the surplus and structural variables of (LP) respectively — giving a direct correspondence between (LP)'s own variables and the surplus variables of Ãx≥b̃. For a basic solution with nonbasic set J (row set M1∪M2 from Chapter 8), Â:=Ã_J is the resulting nonsingular submatrix, and row k of the tableau reads x_k+ Σ_{j∈J}ā_{kj}s_j=ā_{k0}. Adding γ times row i to row k gives the composite row (9.10), x_k+γx_i+Σ_{j∈J}(ā_{kj}+γā_{ij})s_j=ā_{k0}+γā_{i0}, from which a new simple disjunctive cut can be read off whenever 0<ā_{k0}+γā_{i0}<1.

Formalization targets

Theorem 9.3 (goal) — the most-improving pivot column

The pivot column in row i most improving the cut from row k is indexed by l*∈J minimizing f⁺(γ_l) (if ā_{kl}ā_{il}<0) or f⁻(γ_l) (if ā_{kl}ā_{il}>0), over all l∈J with -ā_{k0}/ā_{i0}<γ_l<(1-ā_{k0})/ā_{i0}, γ_l:=-ā_{kl}/ā_{il}.

The chain of results building toward it

Lemma 9.1 (the tableau coefficients' closed form, eq. (9.4)-(9.5)) and Theorem 9.2 (the reduced costs of the CGLP columns u_i,v_i in terms of tableau coefficients, eq. (9.6)) are the two milestones the goal's own machinery is built from. Proposition 9.4, a bridge to Chapters 10-11's general split disjunctions, is included as a genuine milestone despite its payoff lying mostly outside this chapter.

Significance

The results themselves. This chapter is what makes lift-and-project cuts practical: instead of solving an (m+p+n)-row auxiliary LP from scratch for every candidate cut, a single pivot on the (LP)'s own tableau — guided by reduced costs that are themselves closed-form functions of tableau entries — identifies whether an improving cut exists and which one it is. Theorem 9.3's evaluation functions f⁺,f⁻ are exactly the tool a cutting-plane implementation would compute at every candidate pivot.

Formalizing it. No object in this mission exists on the platform prior to it or in Mathlib. This mission restates 08-cut-correspondence's (CGLP)_k apparatus locally, per the series convention and BRIEF.md's explicit instruction, and extends it with this chapter's own generalization to an arbitrary tableau row (needed since Lemma 9.1/Theorem 9.2 concern every basic variable's row, not only the disjunction row k).

Difficulty

Lemma 9.1's book proof is a four-case block-matrix verification (structural/surplus, basic/nonbasic); this mission instead states its content as the identity it is actually for — that the closed-form coefficients express every row's slack as an affine function of the nonbasic rows' slacks, for every point x — which follows tautologically from x=Â⁻¹b̂+Â⁻¹s_J's own definition once stated this way, without needing to reconstruct the block-matrix case analysis. Theorem 9.2's difficulty is that "reduced cost" is not already available as a formalized LP concept in this mission's apparatus; rather than build a generic LP reduced-cost theory, this mission follows the book's own derivation directly — explicitly constructing the pivoted-out extension of a basic solution (eq. (9.7)-(9.9)) and asserting that its objective value decomposes with r_{u_i},r_{v_i} as coefficients, which is genuine, non-circular content matching the proof's own final step ("we can then read the reduced costs... as the coefficients").

Formalization scope

This chapter makes the row/variable identification of Chapters 6-8 fully explicit (N directly indexes the structural variables), but no theorem's own displayed formula in this chunk needs that correspondence beyond what SurplusM's row-general treatment (this chunk's own generalization of 08-cut-correspondence's Surplus) already provides — see MODERATION_NOTES.md for why the S/N/B/R/P/Q block structure is proof machinery, not part of the stated content, throughout.

Theorem 9.3's range condition on γ_l, truncated in BRIEF.md's own excerpt, was completed by reading the PDF directly (confirmed identical to the range derived earlier in the same section): -ā_{k0}/ā_{i0}<γ_l<(1-ā_{k0})/ā_{i0}.

Selected references

  • E. Balas, Disjunctive Programming, Springer, 2018. DOI: 10.1007/978-3-030-00148-3, Chapter 9.
  • E. Balas, M. Perregaard, A precise correspondence between lift-and-project cuts, simple disjunctive cuts, and mixed integer Gomory cuts for 0-1 programming, Mathematical Programming B 94 (2003), 221–245 (cited in the text as [33], the origin of the tableau-pivoting procedure this chapter derives Lemma 9.1 and Theorem 9.2 from).
8 thms2 active usersReviewed
🏆Completed
Optimization·Captain: mikedeng1

An Interactive Weighted Tchebycheff Procedure for Multiple Objective Programming I: In the Finite Case the Augmented Weighted Tchebycheff Program Characterizes the Nondominated SetResearch Paper

Motivation

A decision problem with several conflicting objectives, such as cost, risk and service level, has no single optimum. What it has is a set of nondominated outcomes: those that cannot be improved in one objective without being worsened in another. Interactive methods of multiple objective programming search this set with a decision-maker, and at each step they need a computational device that returns nondominated outcomes and can return any of them.

The classical device, maximizing a weighted sum of the objectives, fails the second requirement. On a nonconvex or discrete outcome set it only reaches the supported nondominated points, those on the boundary of the convex hull, and misses the rest (see, e.g., Boyd and Vandenberghe, Convex Optimization, §4.7.4, where the weighted-sum approach is shown to be sufficient but not necessary for Pareto optimality). Steuer and Choo (Math. Programming 26 (1983) 326–344) replaced the weighted sum by a weighted Tchebycheff distance to an ideal point, augmented by a small linear term. Their procedure became one of the standard interactive methods of the field, and the augmented Tchebycheff scalarization is now a standard tool in multiobjective integer programming and in the generation of nondominated sets.

Timeline, as recorded in the paper's own references. Dinkelbach and Dürr (1972) showed, in the linear case, that among the minimizers of a weighted Tchebycheff program there is always a nondominated one (the paper's Theorem 3.1 extends this to the discrete case). Bowman (Lecture Notes in Economics and Mathematical Systems, as cited by the paper) related the Tchebycheff norm to the efficient frontier of multiple-criteria problems. Choo and Atkins (Computers and Operations Research 7, 1980) and Choo's dissertation (1980) developed interactive weighted Tchebycheff algorithms. Steuer and Choo (1983) added the augmentation term ρ eT(z∗−z)\rho\,e^{\mathsf T}(z^*-z)ρeT(z∗−z), gave an explicit choice of the weights and of ρ\rhoρ in the discrete case, and proved that the resulting program characterizes the nondominated set exactly (Theorem 3.7).

Setting

There are k≥1k \ge 1k≥1 objectives to be maximized. The set of attainable criterion vectors is a finite set Z⊂RkZ \subset \mathbb R^kZ⊂Rk (in the paper, ZZZ is the image of a discrete feasible set SSS under the objectives f1,…,fkf_1,\dots,f_kf1​,…,fk​). A vector zzz dominates zˉ\bar zzˉ if zi≥zˉiz_i \ge \bar z_izi​≥zˉi​ for all iii and zi>zˉiz_i > \bar z_izi​>zˉi​ for at least one iii. The nondominated set N⊆ZN \subseteq ZN⊆Z consists of the zˉ∈Z\bar z \in Zzˉ∈Z that no z∈Zz \in Zz∈Z dominates.

An ideal criterion vector z∗∈Rkz^* \in \mathbb R^kz∗∈Rk has coordinates zi∗=max⁡z∈Zzi+εiz^*_i = \max_{z\in Z} z_i + \varepsilon_izi∗​=maxz∈Z​zi​+εi​ with εi≥0\varepsilon_i \ge 0εi​≥0, where εi\varepsilon_iεi​ must be strictly positive if (i) more than one nondominated vector maximizes objective iii, or (ii) the only nondominated vector maximizing objective iii also maximizes another objective.

Weights range over the simplex Λˉ={λ∈Rk∣λi≥0, ∑iλi=1}\bar\Lambda = \{\lambda \in \mathbb R^k \mid \lambda_i \ge 0,\ \sum_i \lambda_i = 1\}Λˉ={λ∈Rk∣λi​≥0, ∑i​λi​=1}. For a scalar ρ\rhoρ the augmented weighted Tchebycheff program is

min⁡ α+ρ eT(z∗−z)s.t.α≥λi(zi∗−zi), 1≤i≤k,z∈Z,\min\ \alpha + \rho\, e^{\mathsf T}(z^* - z) \quad\text{s.t.}\quad \alpha \ge \lambda_i (z^*_i - z_i),\ 1 \le i \le k,\quad z \in Z,min α+ρeT(z∗−z)s.t.α≥λi​(zi∗​−zi​), 1≤i≤k,z∈Z,

where eee is the vector of ones; at a fixed zzz its value is max⁡iλi(zi∗−zi)+ρ eT(z∗−z)\max_i \lambda_i(z^*_i - z_i) + \rho\,e^{\mathsf T}(z^*-z)maxi​λi​(zi∗​−zi​)+ρeT(z∗−z).

For zp∈Zz^p \in Zzp∈Z the paper defines weights λp\lambda^pλp by (b): λip∝1/(zi∗−zip)\lambda^p_i \propto 1/(z^*_i - z^p_i)λip​∝1/(zi∗​−zip​), normalized to sum to one, when zip≠zi∗z^p_i \ne z^*_izip​=zi∗​ for all iii; otherwise λp\lambda^pλp puts weight 111 on the coordinates where zip=zi∗z^p_i = z^*_izip​=zi∗​ and 000 elsewhere. With αpq=max⁡iλip(zi∗−ziq)\alpha_{pq} = \max_i \lambda^p_i (z^*_i - z^q_i)αpq​=maxi​λip​(zi∗​−ziq​) it sets

ρ=12min⁡zi∈N, zj∈Z{αij−αiieT(zj−zi)  ∣  eT(zj−zi)>0}.(3.8)\rho = \tfrac12 \min_{z^i \in N,\ z^j \in Z}\Big\{\frac{\alpha_{ij} - \alpha_{ii}}{e^{\mathsf T}(z^j - z^i)} \;\Big|\; e^{\mathsf T}(z^j - z^i) > 0\Big\}. \tag{3.8}ρ=21​zi∈N, zj∈Zmin​{eT(zj−zi)αij​−αii​​​eT(zj−zi)>0}.(3.8)

Formalization targets

Goal: Theorem 3.7

For every zp∈Zz^p \in Zzp∈Z,

zp∈N  ⟺  ∃λ∈Λˉ  ∀z∈Z: max⁡iλi(zi∗−zip)+ρ eT(z∗−zp)≤max⁡iλi(zi∗−zi)+ρ eT(z∗−z),z^p \in N \iff \exists \lambda \in \bar\Lambda\ \ \forall z \in Z:\ \max_i \lambda_i(z^*_i - z^p_i) + \rho\, e^{\mathsf T}(z^*-z^p) \le \max_i \lambda_i(z^*_i - z_i) + \rho\, e^{\mathsf T}(z^*-z),zp∈N⟺∃λ∈Λˉ  ∀z∈Z: imax​λi​(zi∗​−zip​)+ρeT(z∗−zp)≤imax​λi​(zi∗​−zi​)+ρeT(z∗−z),

with ρ\rhoρ from (3.8). One coefficient ρ\rhoρ, computed from ZZZ and z∗z^*z∗ alone, works for the whole nondominated set.

Milestones, in the order of the paper

  1. Theorem 3.1. For any λ∈Λˉ\lambda \in \bar\Lambdaλ∈Λˉ, some minimizer of the (unaugmented) weighted Tchebycheff program over ZZZ is nondominated.
  2. Lemma 3.2 (corrected). For zp∈Nz^p \in Nzp∈N and zq∈Zz^q \in Zzq∈Z with zq≠zpz^q \ne z^pzq=zp and zq≰zpz^q \not\le z^pzq≤zp, zqz^qzq lies outside the level set Φ(αpp)\Phi(\alpha_{pp})Φ(αpp​).
  3. Lemma 3.3 (corrected). Under the same hypotheses, αpp<αpq\alpha_{pp} < \alpha_{pq}αpp​<αpq​.
  4. ρp>0\rho_p > 0ρp​>0, the first step of the proof of Theorem 3.4, for the single-vector coefficient ρp\rho_pρp​ of (3.6).
  5. Theorem 3.4. Each zp∈Nz^p \in Nzp∈N is the unique minimizer of the augmented program with weights λp\lambda^pλp and coefficient ρp\rho_pρp​.
  6. Corollary 3.9. The same with the common ρ\rhoρ of (3.8), and λp∈Λˉ\lambda^p \in \bar\Lambdaλp∈Λˉ.

Printed Lemmas 3.2 and 3.3 are false. For Z={(5,3),(5,1),(1,10)}Z = \{(5,3), (5,1), (1,10)\}Z={(5,3),(5,1),(1,10)} and z∗=(5,10)z^* = (5,10)z∗=(5,10), which is ideal with ε=0\varepsilon = 0ε=0, the nondominated vector zp=(5,3)z^p = (5,3)zp=(5,3) has λp=(1,0)\lambda^p = (1,0)λp=(1,0) and αpp=0\alpha_{pp} = 0αpp​=0, while the dominated vector zq=(5,1)z^q = (5,1)zq=(5,1) lies in Φ(0)={z∣z1≥5}\Phi(0) = \{z \mid z_1 \ge 5\}Φ(0)={z∣z1​≥5} and has αpq=0\alpha_{pq} = 0αpq​=0. The proof's second case assumes that only zpz^pzp reaches zj∗z^*_jzj∗​ in coordinate jjj, but the ε\varepsilonε-rule constrains nondominated vectors only. The mission states both lemmas with the added hypothesis zq≰zpz^q \not\le z^pzq≤zp, under which they hold; the milestone texts are the printed ones. Theorems 3.4, 3.7 and Corollary 3.9 are unaffected, since for zq≤zpz^q \le z^pzq≤zp, zq≠zpz^q \ne z^pzq=zp the augmentation term separates zqz^qzq from zpz^pzp on its own. Hypothesis (a) of the paper also contains the misprint "zq≠zqz^q \ne z^qzq=zq" for zq≠zpz^q \ne z^pzq=zp.

Significance

Theorem 3.7 says that the augmented weighted Tchebycheff program, with a computable ρ\rhoρ, is an exact scalarization of the discrete multiple objective program. It returns only nondominated vectors (unlike the plain Tchebycheff program, whose optima can be weakly dominated) and it can return every nondominated vector, including unsupported ones (unlike weighted sums). Corollary 3.9 adds that each nondominated vector is the unique optimum for a suitable weight, so it is found even by a solver that stops at the first optimum. These facts underlie the interactive Tchebycheff procedure of the paper's §5 and a large body of later work on generating nondominated sets of multiobjective integer programs.

The results are proved in the paper; to our knowledge none has been machine-checked. The mission produces a checked version with the two lemmas of the paper's proof chain corrected, a precise treatment of the ideal-vector rule, and an explicit ρ\rhoρ. Alternative proofs, for instance one for the goal that avoids the explicit ρ\rhoρ of (3.8), are welcome.

Difficulty

The ⇐ direction is short. The work is in ⇒: the explicit weights λp\lambda^pλp must be shown to lie in Λˉ\bar\LambdaΛˉ and to make zpz^pzp strictly better than every competitor that is not below it. Both depend on the ε\varepsilonε-rule for z∗z^*z∗, whose role is subtle: it forbids two coordinates of a nondominated vector from reaching z∗z^*z∗, and forbids two nondominated vectors from sharing a coordinate equal to zj∗z^*_jzj∗​, but it says nothing about dominated vectors. The paper's own argument overlooks exactly those dominated vectors, so a proof that follows the printed Lemma 3.2 literally will fail; the gap is closed only by combining the corrected lemma with the augmentation term. Choosing a single ρ\rhoρ for all of NNN also requires that every quotient in (3.8) be strictly positive.

Formalization scope

Criterion vectors are Fin k → ℝ (objective indices 0,…,k−10,\dots,k-10,…,k−1), ZZZ is a Finset, and k≥1k \ge 1k≥1 is imposed as [NeZero k]. The decision set SSS, the objectives fif_ifi​ and the program variable α\alphaα are eliminated: the programs are stated over ZZZ, and α\alphaα is replaced by its minimal value max⁡iλi(zi∗−zi)\max_i \lambda_i(z^*_i - z_i)maxi​λi​(zi∗​−zi​). The programs use zi∗−ziz^*_i - z_izi∗​−zi​ without absolute values, as printed; on ZZZ this equals the metric's ∣zi∗−zi∣|z^*_i - z_i|∣zi∗​−zi​∣ when z∗z^*z∗ is ideal. "zzz minimizes the program" means that zzz minimizes the value over ZZZ, and "uniquely minimizes" means that every other element of ZZZ has a strictly larger value. Λˉ\bar\LambdaΛˉ is Mathlib's stdSimplex ℝ (Fin k).

The ideal vector is encoded with its full ε\varepsilonε-rule, not as "z∗>zz^* > zz∗>z for all z∈Zz \in Zz∈Z"; the latter would exclude the paper's case where zpz^pzp touches z∗z^*z∗ in one coordinate. The minima in (3.6) and (3.8) can range over empty sets (e.g. Z=N={zp}Z = N = \{z^p\}Z=N={zp}); the paper assigns them no value, and the formalization sets ρp\rho_pρp​, ρ\rhoρ to 111 then. A value of 000 would make the goal's ⇒ direction false, so no formalization may rely on Lean's default for an empty minimum. Theorem 3.1 is stated for an arbitrary reference vector z∗z^*z∗, since it needs no ideal-vector hypothesis. The paper's "Let NNN be finite" in Theorem 3.7 is taken as "ZZZ finite", which is what (3.8) and the proof require.

A trivializing formalization would take ρ=0\rho = 0ρ=0 or leave λ\lambdaλ unconstrained; both are excluded, since ρ\rhoρ is the specific value (3.8) and λ\lambdaλ ranges over Λˉ\bar\LambdaΛˉ.

The definitions (dominance, NNN, ideal vector, Tchebycheff values, Φ\PhiΦ, the weights and coefficients of §3) are reusable for the continuous and polyhedral cases of the paper's §4 and for other scalarization results. Contributions of proofs of any milestone are welcome.

Selected references

  • R. E. Steuer and E.-U. Choo, An Interactive Weighted Tchebycheff Procedure for Multiple Objective Programming, Mathematical Programming 26 (1983) 326–344. https://doi.org/10.1007/BF02591870
  • W. Dinkelbach and W. Dürr, Effizienzaussagen bei Ersatzprogrammen zum Vektormaximumproblem, in: R. Henn, H. P. Künzi and H. Schubert (eds.), Operations Research Verfahren XII, Anton Hain, Meisenheim, 1972, 117–123 (reference [4] of the paper; no online version known).
  • V. J. Bowman, On the Relationship of the Tchebycheff Norm and the Efficient Frontier of Multiple-Criteria Objectives, Lecture Notes in Economics and Mathematical Systems, Springer (reference [1] of the paper).
  • E.-U. Choo and D. R. Atkins, An Interactive Algorithm for Multicriteria Programming, Computers and Operations Research 7 (1980) 81–87 (reference [3] of the paper).
  • S. Boyd and L. Vandenberghe, Convex Optimization, Cambridge University Press, 2004, §4.7.4. https://web.stanford.edu/~boyd/cvxbook/
11 thms2 active usersReviewed
🏆Completed
Linear OptimizationOptimization·Captain: Shuze Chen

Disjunctive Programming VIII: Nonlinear Higher-Dimensional RepresentationsTextbook

Motivation

Chapter 2's convex-hull machinery gives an exact, finitely-generated linear description of a disjunctive set's convex hull, but for a mixed 0-1 program with ppp binary variables that description lives in a space with roughly pnpnpn auxiliary variables — one full lift per disjunction. This chapter surveys the alternative nonlinear higher-dimensional constructions that several authors proposed for the same target, conv(K0)\mathrm{conv}(K_0)conv(K0​): multiplying the constraint system by products of xjx_jxj​ and 1−xj1-x_j1−xj​ and linearizing the resulting quadratic terms, rather than disjoining and projecting one variable at a time. Two such constructions — Lovász and Schrijver's "cones of matrices" lift N(K)N(K)N(K), and Sherali and Adams's hierarchy KtK_tKt​ — both converge to the integer hull, and the chapter's central point is that both convergence proofs reduce, after all the nonlinear machinery is stripped away, to results already established by disjunctive programming's own one-variable-at-a-time convexification (Theorem 2.1, specialized here to Theorem 7.1, and the sequential-convexifiability theorem of Chapter 3).

Setting

K:={x∈Rn:Ax≥b, x≥0, xj≤1, j=1,…,p}={x:A~x≥b~}K := \{x \in \mathbb R^n : Ax \ge b,\ x \ge 0,\ x_j \le 1,\ j=1,\dots,p\} = \{x : \tilde A x \ge \tilde b\}K:={x∈Rn:Ax≥b, x≥0, xj​≤1, j=1,…,p}={x:A~x≥b~} is the LP relaxation of a mixed 0-1 program with ppp of its nnn variables 0-1 constrained, and K0:=K∩{xj∈{0,1}, j=1,…,p}K_0 := K \cap \{x_j \in \{0,1\},\ j=1,\dots,p\}K0​:=K∩{xj​∈{0,1}, j=1,…,p} its feasible set. Pj(K)P_j(K)Pj​(K) (Section 7.1) multiplies A~x≥b~\tilde A x \ge \tilde bA~x≥b~ by (1−xj)(1-x_j)(1−xj​) and xjx_jxj​, linearizes yi:=xixjy_i := x_ix_jyi​:=xi​xj​ and xj:=xj2x_j := x_j^2xj​:=xj2​, and projects onto xxx; iterating over a coordinate sequence gives Pi1,…,it(K)P_{i_1,\dots,i_t}(K)Pi1​,…,it​​(K). N(K)N(K)N(K) (Section 7.2, Lovász-Schrijver) instead linearizes with a single symmetric matrix YYY (Yij=Yji=xixjY_{ij} = Y_{ji} = x_ix_jYij​=Yji​=xi​xj​ for every pair) before projecting, and iterates as Nt(K):=N(Nt−1(K))N^t(K) := N(N^{t-1}(K))Nt(K):=N(Nt−1(K)). KtK_tKt​ (Section 7.3, Sherali-Adams) multiplies by every product of ttt literals ∏j∈J1xj∏j∈J2(1−xj)\prod_{j\in J_1}x_j\prod_{j\in J_2}(1-x_j)∏j∈J1​​xj​∏j∈J2​​(1−xj​) (∣J1∪J2∣=t|J_1\cup J_2|=t∣J1​∪J2​∣=t), linearizes each resulting monomial with a fresh "moment" variable, and projects.

Formalization targets

Theorem 7.6 (goal) — the Sherali-Adams hierarchy reaches the integer hull

Kp=conv(K0).K_p = \mathrm{conv}(K_0).Kp​=conv(K0​).

The chain of results building toward it

Theorem 7.1 (Pj(K)P_j(K)Pj​(K) equals the one-variable convex hull, a special case of Theorem 2.1), Theorem 7.2 (iterating PjP_jPj​ over a fixed sequence reaches the hull of imposing 0/10/10/1 on all of them), Corollary 7.3 (iterating over every 0-1 index reaches conv(K0)\mathrm{conv}(K_0)conv(K0​)), Theorem 7.4 (N(K)⊆Pj(K)N(K) \subseteq P_j(K)N(K)⊆Pj​(K) for every jjj), Theorem 7.5 (iterating NNN over ppp steps reaches conv(K0)\mathrm{conv}(K_0)conv(K0​), by the same containment), and Theorem 7.7 (Kt⊆P1,…,t(K)K_t \subseteq P_{1,\dots,t}(K)Kt​⊆P1,…,t​(K), proved by a genuine induction re-deriving every valid inequality of P1,…,t(K)P_{1,\dots,t}(K)P1,…,t​(K) from (NLt)(NL_t)(NLt​)'s own rows).

Significance

The results themselves. This chapter is disjunctive programming's account of why three independently-developed convexification hierarchies — its own lift-and-project, Lovász-Schrijver, and Sherali-Adams — all reach the same integer hull: not by coincidence, but because each one's convergence proof is, at bottom, a disguised instance of the book's own Theorem 2.1 and Chapter 3 machinery. This is part of what situates disjunctive programming as the unifying framework behind several major lift-and-project hierarchies used throughout integer programming.

Formalizing it. No object in this mission exists on the platform prior to it or in Mathlib. This mission restates 03-sequential-convex's and 02a-convex-hull's vocabulary locally (per the series convention that a draft mission cannot import another draft mission's definitions), specialized throughout to the split disjunction xj∈{0,1}x_j \in \{0,1\}xj​∈{0,1}.

Difficulty

The chapter's own account of the Sherali-Adams construction (Section 7.3) is narrative rather than displaying an explicit linear system for (NLt)(NL_t)(NLt​), unlike every other construction in this chapter (contrast eq. (7.1) and eq. (7.4), both displayed explicitly) — Step 1 says only "multiply A~x≥b~\tilde Ax \ge \tilde bA~x≥b~ with every product of the form ∏j∈J1xj⋅∏j∈J2(1−xj)\prod_{j\in J_1}x_j \cdot \prod_{j\in J_2}(1-x_j)∏j∈J1​​xj​⋅∏j∈J2​​(1−xj​)." Deriving the actual linear system this multiplication produces requires expanding every (1−xj)(1-x_j)(1−xj​) factor via inclusion-exclusion over S⊆J2S \subseteq J_2S⊆J2​ before the monomials can be linearized — a real, if mechanical, derivation step this mission had to carry out itself (RowNLt, see MODERATION_NOTES.md) rather than transcribe from a displayed equation. Theorem 7.7's own proof is a genuine argument (an induction re-deriving a valid inequality of P1,…,t(K)P_{1,\dots,t}(K)P1,…,t​(K) from (NLt)(NL_t)(NLt​)'s rows layer by layer), not a restatement, so it is included as a milestone with real mathematical content rather than assumed.

Correcting BRIEF.md. The brief's "Recommended goal theorem" section mislabels Theorem 7.6 as the Lovász-Schrijver result "Kp=conv(K0)K^p = \mathrm{conv}(K_0)Kp=conv(K0​)" — cross-checked directly against the PDF, Theorem 7.6 (p. 95, PDF 101) is stated [112] Kp = conv(K0), cited to Sherali and Adams, appearing immediately after Section 7.3 introduces KtK_tKt​. The actual Lovász-Schrijver iteration-reaches-the-hull result, cited [99], is Theorem 7.5 (Np(K) = conv(K0)), formalized in this mission as a milestone (lovasz_schrijver_reaches_hull) rather than the goal. See STATUS.md.

Formalization scope

Three distinct convexification operators are kept fully distinguishable throughout, as BRIEF.md warns: Pj/IteratedSplit (one-variable-at-a-time, Section 7.1, Def_..._Basic), NOp/MK (Lovász-Schrijver, Section 7.2, Def_..._Lifts), and KtSet/IsXt (Sherali-Adams, Section 7.3, Def_..._Lifts) — no shared abbreviation or lemma conflates their defining predicates, even though all three converge to the same hull.

Nt(K)N^t(K)Nt(K)'s iteration (Theorem 7.5) is formalized via a dependent family of representations A t, b t for t : Fin (p+1) with a hypothesis relating consecutive steps to NOp's image at the previous step — the same pattern 04-normal-forms's Theorem 4.10 uses — since each successive Nt(K)N^t(K)Nt(K) genuinely has a different, larger ambient constraint system, not a fixed matrix's power.

Theorem 7.7 is stated for an arbitrary ttt-subset S⊆N′S \subseteq N'S⊆N′ rather than the literal prefix {1,…,t}\{1,\dots,t\}{1,…,t} the book's own statement uses (its own proof, by induction, treats an arbitrary inequality of P1,…,t(K)P_{1,\dots,t}(K)P1,…,t​(K) with no dependence on the prefix's specific ordering), which is what the goal theorem's proof, applied at S=N′S = N'S=N′, actually needs.

The Bienstock-Zuckerberg results quoted narratively in Section 7.5 ("Theorem 1"/"Theorem 2", [43]/[44]) are out-of-cone: the book itself presents them only as a survey of a further lift operator, under their own source papers' numbering, not as Balas's own numbered results, and does not restate their proofs.

Selected references

  • E. Balas, Disjunctive Programming, Springer, 2018. DOI: 10.1007/978-3-030-00148-3, Chapter 7.
  • L. Lovász, A. Schrijver, Cones of matrices and set-functions and 0-1 optimization, SIAM Journal on Optimization 1 (1991), 166-190 (cited in the text as [99], the origin of the N(K)N(K)N(K) construction and Theorems 7.4-7.5).
  • H.D. Sherali, W.P. Adams, A hierarchy of relaxations between the continuous and convex hull representations for zero-one programming problems, SIAM Journal on Discrete Mathematics 3 (1990), 411-430 (cited in the text as [112], the origin of the KtK_tKt​ construction and Theorem 7.6).
9 thms2 active usersReviewed
🏆Completed
Linear OptimizationMachine LearningOptimal Transport+2·Captain: mikedeng1

Distributionally Robust Logistic Regression II: Worst- and Best-Case Misclassification Risks over a Wasserstein Ball Are Linear ProgramsResearch Paper

Motivation

A logistic regression model is fitted on finitely many samples, and the quantity a practitioner cares about is the misclassification risk of the fitted classifier on new data. Its empirical counterpart, the training error, is biased downwards, and classical generalization bounds give it an additive margin that depends on a complexity measure of the model class rather than on the data at hand.

Shafieezadeh-Abadeh, Mohajerin Esfahani and Kuhn (NIPS 2015) take a distributionally robust route. They surround the empirical distribution of the training data by a ball of distributions in the Wasserstein metric and, for a given weight vector, compute the largest and the smallest misclassification probability over that ball. Their Theorem 3 shows that both extremes are optimal values of explicit linear programs. Combined with a measure-concentration result for the empirical distribution in the Wasserstein metric (Fournier and Guillin, PTRF 2015), the two values bracket the true risk with a prescribed confidence. The same Wasserstein-ball construction underlies the data-driven optimization framework of Mohajerin Esfahani and Kuhn (Math. Program. 2018).

Setting

Let VVV be the feature space Rn\mathbb R^nRn with an arbitrary norm ∥⋅∥\|\cdot\|∥⋅∥, and let labels take the values y∈{−1,+1}y\in\{-1,+1\}y∈{−1,+1}. The feature-label space is Ξ=V×{−1,+1}\Xi = V\times\{-1,+1\}Ξ=V×{−1,+1} with points ξ=(x,y)\xi=(x,y)ξ=(x,y). A weight vector β\betaβ acts on features by x↦⟨β,x⟩x\mapsto\langle\beta,x\ranglex↦⟨β,x⟩; its dual norm is ∥β∥∗=sup⁡∥x∥≤1⟨β,x⟩\|\beta\|_* = \sup_{\|x\|\le1}\langle\beta,x\rangle∥β∥∗​=sup∥x∥≤1​⟨β,x⟩.

Metric (Definition 2). For a weight κ>0\kappa>0κ>0,

d((x,y),(x′,y′))=∥x−x′∥+κ ∣y−y′∣/2.d\big((x,y),(x',y')\big) = \|x-x'\| + \kappa\,|y-y'|/2 .d((x,y),(x′,y′))=∥x−x′∥+κ∣y−y′∣/2.

Changing a label costs κ\kappaκ; moving a feature costs its norm distance.

Wasserstein distance (Definition 1). For distributions Q,P\mathbb Q,\mathbb PQ,P on Ξ\XiΞ, W(Q,P)W(\mathbb Q,\mathbb P)W(Q,P) is the infimum of ∫d(ξ,ξ′) Π(dξ,dξ′)\int d(\xi,\xi')\,\Pi(d\xi,d\xi')∫d(ξ,ξ′)Π(dξ,dξ′) over all couplings Π\PiΠ of Q\mathbb QQ and P\mathbb PP. The Wasserstein ball of radius ε≥0\varepsilon\ge0ε≥0 is Bε(P)={Q:W(Q,P)≤ε}\mathbb B_\varepsilon(\mathbb P) = \{\mathbb Q : W(\mathbb Q,\mathbb P)\le\varepsilon\}Bε​(P)={Q:W(Q,P)≤ε}.

Data. Training samples (x^i,y^i)(\hat x_i,\hat y_i)(x^i​,y^​i​), i=1,…,Ni=1,\dots,Ni=1,…,N, define the empirical distribution P^N=1N∑i=1Nδ(x^i,y^i)\hat{\mathbb P}_N = \frac1N\sum_{i=1}^N\delta_{(\hat x_i,\hat y_i)}P^N​=N1​∑i=1N​δ(x^i​,y^​i​)​.

Classifier and risk. Logistic regression models Prob⁡(y∣x)=[1+exp⁡(−y⟨β,x⟩)]−1\operatorname{Prob}(y\mid x) = [1+\exp(-y\langle\beta,x\rangle)]^{-1}Prob(y∣x)=[1+exp(−y⟨β,x⟩)]−1 (eq. (1)). The classifier is fβ(x)=+1f_\beta(x)=+1fβ​(x)=+1 if Prob⁡(+1∣x)>0.5\operatorname{Prob}(+1\mid x)>0.5Prob(+1∣x)>0.5 and −1-1−1 otherwise, and its risk under the data-generating distribution P\mathbb PP is R(β)=P[y≠fβ(x)]\mathfrak R(\beta) = \mathbb P[y\ne f_\beta(x)]R(β)=P[y=fβ​(x)].

Worst- and best-case risks.

Rmax⁡(β)=sup⁡Q∈Bε(P^N)EQ[1{y⟨β,x⟩≤0}],Rmin⁡(β)=inf⁡Q∈Bε(P^N)EQ[1{y⟨β,x⟩<0}].\mathfrak R_{\max}(\beta) = \sup_{\mathbb Q\in\mathbb B_\varepsilon(\hat{\mathbb P}_N)}\mathbb E^{\mathbb Q}\big[\mathbb 1_{\{y\langle\beta,x\rangle\le0\}}\big],\qquad \mathfrak R_{\min}(\beta) = \inf_{\mathbb Q\in\mathbb B_\varepsilon(\hat{\mathbb P}_N)}\mathbb E^{\mathbb Q}\big[\mathbb 1_{\{y\langle\beta,x\rangle<0\}}\big].Rmax​(β)=Q∈Bε​(P^N​)sup​EQ[1{y⟨β,x⟩≤0}​],Rmin​(β)=Q∈Bε​(P^N​)inf​EQ[1{y⟨β,x⟩<0}​].

The worst case counts a nonpositive margin, the best case a strictly negative one.

The linear programs. For data (x^i,y^i)(\hat x_i,\hat y_i)(x^i​,y^​i​), a weight vector β^\hat\betaβ^​ and variables λ∈R\lambda\in\mathbb Rλ∈R, s,r,t∈RNs,r,t\in\mathbb R^Ns,r,t∈RN, program (10a) minimizes λε+1N∑isi\lambda\varepsilon + \frac1N\sum_i s_iλε+N1​∑i​si​ subject to, for every iii,

1−riy^i⟨β^,x^i⟩≤si,1+tiy^i⟨β^,x^i⟩−λκ≤si,ri∥β^∥∗≤λ,ti∥β^∥∗≤λ,ri,ti,si≥0.1 - r_i\hat y_i\langle\hat\beta,\hat x_i\rangle\le s_i,\quad 1 + t_i\hat y_i\langle\hat\beta,\hat x_i\rangle - \lambda\kappa\le s_i,\quad r_i\|\hat\beta\|_*\le\lambda,\quad t_i\|\hat\beta\|_*\le\lambda,\quad r_i,t_i,s_i\ge0 .1−ri​y^​i​⟨β^​,x^i​⟩≤si​,1+ti​y^​i​⟨β^​,x^i​⟩−λκ≤si​,ri​∥β^​∥∗​≤λ,ti​∥β^​∥∗​≤λ,ri​,ti​,si​≥0.

Program (10b) has the same objective and bounds, with the signs of the two margin terms exchanged.

Formalization targets

Goal: Theorem 3 (i)–(ii)

For every κ>0\kappa>0κ>0, ε≥0\varepsilon\ge0ε≥0, N≥1N\ge1N≥1, all samples and every weight vector β^\hat\betaβ^​, both programs attain their minima vvv and www, and

Rmax⁡(β^)=v,Rmin⁡(β^)=1−w.\mathfrak R_{\max}(\hat\beta) = v,\qquad \mathfrak R_{\min}(\hat\beta) = 1-w .Rmax​(β^​)=v,Rmin​(β^​)=1−w.

The identities hold for each fixed β^\hat\betaβ^​, so they apply to any β^\hat\betaβ^​ computed from the data.

Milestone: Theorem 3(i) alone

Rmax⁡(β^)\mathfrak R_{\max}(\hat\beta)Rmax​(β^​) equals the minimum of (10a).

Milestones: the confidence clauses

If the training samples are i.i.d. from P\mathbb PP and the radius is such that PN{P∈Bε(P^N)}≥1−η\mathbb P^N\{\mathbb P\in\mathbb B_\varepsilon(\hat{\mathbb P}_N)\}\ge1-\etaPN{P∈Bε​(P^N​)}≥1−η, then for any sample-dependent β^\hat\betaβ^​

PN{R(β^)≤Rmax⁡(β^)}≥1−η,PN{Rmin⁡(β^)≤R(β^)}≥1−η,\mathbb P^N\{\mathfrak R(\hat\beta)\le\mathfrak R_{\max}(\hat\beta)\}\ge1-\eta,\qquad \mathbb P^N\{\mathfrak R_{\min}(\hat\beta)\le\mathfrak R(\hat\beta)\}\ge1-\eta,PN{R(β^​)≤Rmax​(β^​)}≥1−η,PN{Rmin​(β^​)≤R(β^​)}≥1−η, PN{Rmin⁡(β^)≤R(β^)≤Rmax⁡(β^)}≥1−2η.\mathbb P^N\{\mathfrak R_{\min}(\hat\beta)\le\mathfrak R(\hat\beta)\le\mathfrak R_{\max}(\hat\beta)\}\ge1-2\eta .PN{Rmin​(β^​)≤R(β^​)≤Rmax​(β^​)}≥1−2η.

Significance

The result. Theorem 3 replaces an optimization over an infinite-dimensional set of distributions by a linear program with 3N+13N+13N+1 variables and 4N4N4N constraints plus sign constraints. That makes the worst- and best-case misclassification probabilities computable at the scale of the training set, for any norm on the features whose dual norm can be evaluated. With the confidence clauses, the two values are data-driven upper and lower confidence bounds on the out-of-sample risk of the classifier actually deployed, including one fitted on the same data.

Formalizing it. The paper states Theorem 3 without proof in the main text; the argument is deferred to a technical appendix. No part of it is machine-checked. A formal proof needs the evaluation of a worst-case probability of a closed set over a type-1 Wasserstein ball around a discrete distribution, and the analogous best-case probability of an open set. Both are reusable in any Wasserstein-robust treatment of chance constraints or classification error.

Difficulty

The objective 1{y⟨β,x⟩≤0}\mathbf 1_{\{y\langle\beta,x\rangle\le0\}}1{y⟨β,x⟩≤0}​ is neither continuous nor concave, so the duality theorems for Wasserstein balls stated for continuous or Lipschitz losses do not apply directly. Upper semicontinuity of the indicator of a closed set is what matters, and the strict inequality in Rmin⁡\mathfrak R_{\min}Rmin​ has to be handled as the complement of a closed set. The transport cost couples a norm on the features with a discrete label-flip cost, so a sample can reach the misclassification region either by moving its feature to the hyperplane ⟨β^,x⟩=0\langle\hat\beta,x\rangle=0⟨β^​,x⟩=0 or by flipping its label, and the two options interact through the shared budget ε\varepsilonε. Distances to the hyperplane are measured in the given norm and produce the dual norm ∥β^∥∗\|\hat\beta\|_*∥β^​∥∗​. The degenerate weight β^=0\hat\beta=0β^​=0 (every point on the hyperplane) must come out correctly without any division by ∥β^∥∗\|\hat\beta\|_*∥β^​∥∗​.

Formalization scope

The feature space is a finite-dimensional real normed space V with an arbitrary norm, standing for (Rn,∥⋅∥)(\mathbb R^n,\|\cdot\|)(Rn,∥⋅∥); the Euclidean norm is not assumed. A weight vector is a continuous linear functional V →L[ℝ] ℝ, and ∥β^∥∗\|\hat\beta\|_*∥β^​∥∗​ is its operator norm, which is exactly the dual norm. Labels are Bool with an explicit embedding true↦+1\text{true}\mapsto+1true↦+1, false↦−1\text{false}\mapsto-1false↦−1; the metric of Definition 2 is written literally. The Wasserstein distance is ℝ≥0∞-valued, probabilities and expectations of indicators are measure values in [0,∞][0,\infty][0,∞], and suprema and infima range exactly over the probability measures in the ball. "min" in (10a)/(10b) is formalized as attainment (IsLeast) of the objective over the feasible set. Samples are indexed by Fin N with N≥1N\ge1N≥1.

The following choices differ from a literal reading of the page:

  • The paper says the risk "can be expressed as" EP[1{y⟨β,x⟩≤0}]\mathbb E^{\mathbb P}[\mathbb 1_{\{y\langle\beta,x\rangle\le0\}}]EP[1{y⟨β,x⟩≤0}​]. This fails on the hyperplane ⟨β,x⟩=0\langle\beta,x\rangle=0⟨β,x⟩=0, where fβ(x)=−1f_\beta(x)=-1fβ​(x)=−1 is correct for y=−1y=-1y=−1. The mission defines R(β)=P[y≠fβ(x)]\mathfrak R(\beta)=\mathbb P[y\ne f_\beta(x)]R(β)=P[y=fβ​(x)] from (1) and includes the true statement EP[1{y⟨β,x⟩<0}]≤R(β)≤EP[1{y⟨β,x⟩≤0}]\mathbb E^{\mathbb P}[\mathbb 1_{\{y\langle\beta,x\rangle<0\}}]\le\mathfrak R(\beta)\le\mathbb E^{\mathbb P}[\mathbb 1_{\{y\langle\beta,x\rangle\le0\}}]EP[1{y⟨β,x⟩<0}​]≤R(β)≤EP[1{y⟨β,x⟩≤0}​] as a helper item.
  • The choice ε=εN(η)\varepsilon=\varepsilon_N(\eta)ε=εN​(η) of (8) and the measure-concentration theorem behind it (Theorem 2) are not formalized. The confidence clauses take their conclusion, PN{P∈Bε(P^N)}≥1−η\mathbb P^N\{\mathbb P\in\mathbb B_\varepsilon(\hat{\mathbb P}_N)\}\ge1-\etaPN{P∈Bε​(P^N​)}≥1−η, as a hypothesis, and "with probability 1−η1-\eta1−η" is read as "with probability at least 1−η1-\eta1−η". The printed level 1−2η1-2\eta1−2η is kept for the two-sided bound.

Swapping the strict and non-strict inequalities in Rmax⁡\mathfrak R_{\max}Rmax​ and Rmin⁡\mathfrak R_{\min}Rmin​, restricting the supremum to measures supported on the sample points, or replacing the ball by a set that excludes non-discrete distributions would each change the theorem. None of these is an acceptable reformulation of the goal.

Useful infrastructure: couplings of a discrete measure with an arbitrary one, the distance from a point to a closed half-space in a general norm, and LP-duality arguments for fractional-knapsack-type programs. Proofs of the helper and confidence items, and any reusable lemma about worst-case probabilities of closed sets over Wasserstein balls, are welcome.

Selected references

  • S. Shafieezadeh-Abadeh, P. Mohajerin Esfahani, D. Kuhn, Distributionally Robust Logistic Regression, Advances in Neural Information Processing Systems 28 (NIPS 2015). https://papers.nips.cc/paper/2015/hash/cc1aa436277138f61cda703991069eaf-Abstract.html
  • N. Fournier, A. Guillin, On the rate of convergence in Wasserstein distance of the empirical measure, Probability Theory and Related Fields 162 (2015). https://doi.org/10.1007/s00440-014-0583-7
  • P. Mohajerin Esfahani, D. Kuhn, Data-driven distributionally robust optimization using the Wasserstein metric: performance guarantees and tractable reformulations, Mathematical Programming 171 (2018). https://doi.org/10.1007/s10107-017-1172-1
8 thms2 active usersReviewed
🏆Completed
Convex OptimizationMachine LearningOptimal Transport+1·Captain: mikedeng1

Distributionally Robust Logistic Regression I: The Worst-Case Expected Logloss over a Wasserstein Ball Is a Tractable Convex ProgramResearch Paper

Motivation

Logistic regression is among the most widely used classification methods in statistics and machine learning. Its maximum-likelihood estimator minimizes the average logloss on the training data and is known to overfit when data are scarce; practitioners respond with ad hoc regularization, typically a norm penalty on the weight vector. Shafieezadeh-Abadeh, Mohajerin Esfahani and Kuhn (NIPS 2015, arXiv:1509.09259) replace the empirical average by a worst case over all distributions within a Wasserstein ball around the empirical distribution. The resulting model has a finite convex reformulation, contains classical and norm-regularized logistic regression as special cases, and comes with out-of-sample guarantees. It is one of the early instances of Wasserstein distributionally robust optimization in learning, building on the duality theory of Mohajerin Esfahani and Kuhn (Math. Program. 2018, arXiv:1505.05116); the regularization interpretation was later extended to general losses by Shafieezadeh-Abadeh, Kuhn and Mohajerin Esfahani (JMLR 2019, arXiv:1710.10016).

Setting

Let VVV be the feature space Rn\mathbb R^nRn with an arbitrary norm ∥⋅∥\|\cdot\|∥⋅∥, and let ∥β∥∗=sup⁡∥x∥≤1⟨β,x⟩\|\beta\|_* = \sup_{\|x\|\le1}\langle\beta,x\rangle∥β∥∗​=sup∥x∥≤1​⟨β,x⟩ be the dual norm of a weight vector β\betaβ. Labels are y∈{−1,+1}y\in\{-1,+1\}y∈{−1,+1}, and the feature-label space is Ξ=V×{−1,+1}\Xi = V\times\{-1,+1\}Ξ=V×{−1,+1}. The logloss of β\betaβ at (x,y)(x,y)(x,y) is

lβ(x,y)=log⁡(1+exp⁡(−y⟨β,x⟩)).l_\beta(x,y) = \log\big(1+\exp(-y\langle\beta,x\rangle)\big).lβ​(x,y)=log(1+exp(−y⟨β,x⟩)).

For a label weight κ>0\kappa>0κ>0, the metric of Definition 2 on Ξ\XiΞ is

d((x,y),(x′,y′))=∥x−x′∥+κ ∣y−y′∣/2,d\big((x,y),(x',y')\big) = \|x-x'\| + \kappa\,|y-y'|/2 ,d((x,y),(x′,y′))=∥x−x′∥+κ∣y−y′∣/2,

so that changing a label costs κ\kappaκ. The Wasserstein distance W(Q,P)W(\mathbb Q,\mathbb P)W(Q,P) between probability distributions on Ξ\XiΞ (Definition 1) is the infimum of ∫d(ξ,ξ′) Π(dξ,dξ′)\int d(\xi,\xi')\,\Pi(d\xi,d\xi')∫d(ξ,ξ′)Π(dξ,dξ′) over all couplings Π\PiΠ of Q\mathbb QQ and P\mathbb PP, and Bε(P)={Q:W(Q,P)≤ε}\mathbb B_\varepsilon(\mathbb P) = \{\mathbb Q : W(\mathbb Q,\mathbb P)\le\varepsilon\}Bε​(P)={Q:W(Q,P)≤ε}. Given training samples (x^i,y^i)i=1N(\hat x_i,\hat y_i)_{i=1}^N(x^i​,y^​i​)i=1N​, the empirical distribution is P^N=1N∑iδ(x^i,y^i)\hat{\mathbb P}_N = \frac1N\sum_i\delta_{(\hat x_i,\hat y_i)}P^N​=N1​∑i​δ(x^i​,y^​i​)​, and the distributionally robust logistic regression problem (6) is

J^=inf⁡β sup⁡Q∈Bε(P^N)EQ[lβ(x,y)].\hat J = \inf_\beta\ \sup_{\mathbb Q\in\mathbb B_\varepsilon(\hat{\mathbb P}_N)} \mathbb E^{\mathbb Q}\big[l_\beta(x,y)\big].J^=βinf​ Q∈Bε​(P^N​)sup​EQ[lβ​(x,y)].

Program (7) has variables β\betaβ, λ∈R\lambda\in\mathbb Rλ∈R, s∈RNs\in\mathbb R^Ns∈RN, objective λε+1N∑isi\lambda\varepsilon + \frac1N\sum_i s_iλε+N1​∑i​si​, and constraints lβ(x^i,y^i)≤sil_\beta(\hat x_i,\hat y_i)\le s_ilβ​(x^i​,y^​i​)≤si​, lβ(x^i,−y^i)−λκ≤sil_\beta(\hat x_i,-\hat y_i)-\lambda\kappa\le s_ilβ​(x^i​,−y^​i​)−λκ≤si​ for all iii, and ∥β∥∗≤λ\|\beta\|_*\le\lambda∥β∥∗​≤λ.

Formalization targets

Goal: Theorem 1 (tractable reformulation)

For every ε≥0\varepsilon\ge0ε≥0, κ>0\kappa>0κ>0, N≥1N\ge1N≥1 and every norm on the feature space,

inf⁡β sup⁡Q∈Bε(P^N)EQ[lβ]  =  inf⁡{λε+1N∑isi:(β,λ,s) feasible for (7)},\inf_\beta\ \sup_{\mathbb Q\in\mathbb B_\varepsilon(\hat{\mathbb P}_N)}\mathbb E^{\mathbb Q}[l_\beta] \;=\; \inf\Big\{\lambda\varepsilon+\tfrac1N\textstyle\sum_i s_i : (\beta,\lambda,s)\text{ feasible for (7)}\Big\},βinf​ Q∈Bε​(P^N​)sup​EQ[lβ​]=inf{λε+N1​∑i​si​:(β,λ,s) feasible for (7)},

and for ε>0\varepsilon>0ε>0 the infimum of (7) is attained.

Milestones

  1. §3.1 — the feasible set of (7) is convex.
  2. §2 — for ε=0\varepsilon=0ε=0 the worst-case expected logloss is the empirical average logloss, so (6) reduces to classical logistic regression (2).
  3. Theorem 1 for fixed β\betaβ — sup⁡Q∈Bε(P^N)EQ[lβ]\sup_{\mathbb Q\in\mathbb B_\varepsilon(\hat{\mathbb P}_N)}\mathbb E^{\mathbb Q}[l_\beta]supQ∈Bε​(P^N​)​EQ[lβ​] equals the attained minimum of (7) over (λ,s)(\lambda,s)(λ,s) with β\betaβ fixed.
  4. Remark 2, eq. (9) — at an optimal solution (β^,λ^,s^)(\hat\beta,\hat\lambda,\hat s)(β^​,λ^,s^),
J^=λ^ε+EP^N[lβ^]+1N∑imax⁡{0,y^i⟨β^,x^i⟩−λ^κ}.\hat J = \hat\lambda\varepsilon + \mathbb E^{\hat{\mathbb P}_N}[l_{\hat\beta}] + \tfrac1N\textstyle\sum_i\max\{0,\hat y_i\langle\hat\beta,\hat x_i\rangle-\hat\lambda\kappa\}.J^=λ^ε+EP^N​[lβ^​​]+N1​∑i​max{0,y^​i​⟨β^​,x^i​⟩−λ^κ}.
  1. Remark 1 — as κ→∞\kappa\to\inftyκ→∞ the optimal value of (7) converges to inf⁡βε∥β∥∗+1N∑ilβ(x^i,y^i)\inf_\beta \varepsilon\|\beta\|_* + \frac1N\sum_i l_\beta(\hat x_i,\hat y_i)infβ​ε∥β∥∗​+N1​∑i​lβ​(x^i​,y^​i​).
  2. Theorem 2, implication — if PN{P∈Bε(P^N)}≥1−η\mathbb P^N\{\mathbb P\in\mathbb B_\varepsilon(\hat{\mathbb P}_N)\}\ge1-\etaPN{P∈Bε​(P^N​)}≥1−η, then PN{EP[lβ^]≤J^}≥1−η\mathbb P^N\{\mathbb E^{\mathbb P}[l_{\hat\beta}]\le\hat J\}\ge1-\etaPN{EP[lβ^​​]≤J^}≥1−η.

Significance

Theorem 1 turns a minimax problem over an infinite-dimensional family of distributions into a finite convex program whose size grows linearly in NNN; with the ℓ1\ell_1ℓ1​, ℓ2\ell_2ℓ2​ or ℓ∞\ell_\inftyℓ∞​ norm it is a standard exponential-cone or conic program. Remark 1 explains norm-regularized logistic regression as a distributionally robust model: the regularizer is the dual norm of the transport cost on features, and the regularization weight is the radius of the ambiguity set. Remark 2 exposes an additional term that accounts for label noise and vanishes as label changes become prohibitively expensive. Theorem 2 makes the optimal value J^\hat JJ^ a certificate on the out-of-sample logloss whenever the ball contains the true distribution.

The paper's proofs are in a technical appendix and have not been machine-checked. Mathlib contains no Wasserstein distributionally robust duality. This mission produces a formal statement of the reformulation with an arbitrary norm and a label-dependent cost, together with formal versions of the paper's printed consequences of it (Remarks 1 and 2, the ε=0\varepsilon=0ε=0 reduction, and the implication in Theorem 2).

Difficulty

The worst-case expectation ranges over every Borel probability distribution within transport distance ε\varepsilonε of the empirical distribution, including distributions with unbounded support and distributions that move mass across labels. Exhibiting good distributions in the ball shows only that the robust value is at least the value of (7); the reverse inequality must control every distribution in the ball at once, and nothing in the definition of the ball bounds its elements' supports. The obvious simplification, restricting attention to distributions supported on finitely many points, again yields only a one-sided bound unless the supremum is shown to be approached by such distributions. The label term of the metric couples the two label classes, so results for a pure norm cost on the features do not apply directly, and the dual norm enters through an arbitrary norm rather than the Euclidean one.

Formalization scope

  • The feature space is an abstract finite-dimensional real normed space V standing for (Rn,∥⋅∥)(\mathbb R^n,\|\cdot\|)(Rn,∥⋅∥) with an arbitrary norm; weights are continuous linear functionals V →L[ℝ] ℝ, and ∥β∥∗\|\beta\|_*∥β∥∗​ is their operator norm, which is exactly the dual norm. Labels are Bool, embedded as ±1\pm1±1; the label −y-y−y is Boolean negation. The metric of Definition 2 is written literally.
  • The Wasserstein distance is of type 1, valued in [0,∞][0,\infty][0,∞], with couplings ranging over all probability measures on Ξ×Ξ\Xi\times\XiΞ×Ξ with the two prescribed marginals. The ball consists of probability measures.
  • Expectations of the positive logloss are lower Lebesgue integrals in [0,∞][0,\infty][0,∞], and the supremum over the ball is taken there; the optimal value of (7) is the infimum of its (nonnegative) objective over the feasible set, also in [0,∞][0,\infty][0,∞]. A Bochner integral, which vanishes on non-integrable functions, would make the worst case trivially finite and is not used.
  • The standing hypotheses are κ>0\kappa>0κ>0, ε≥0\varepsilon\ge0ε≥0, N≥1N\ge1N≥1.
  • Correction. The paper prints "min" in (7) for all ε≥0\varepsilon\ge0ε≥0. At ε=0\varepsilon=0ε=0 the minimum can fail to be attained (V=RV=\mathbb RV=R, N=1N=1N=1, x^1=1\hat x_1=1x^1​=1, y^1=+1\hat y_1=+1y^​1​=+1: the value is 000 but every feasible point has positive objective). The goal states the value identity for ε≥0\varepsilon\ge0ε≥0 and attainment for ε>0\varepsilon>0ε>0.
  • Remark 1 is formalized as convergence of optimal values as κ→∞\kappa\to\inftyκ→∞; a metric with κ=∞\kappa=\inftyκ=∞ is not formalized. Only convexity, not tractability, of (7) is stated. The first claim of Theorem 2 (the radius (8) and the light-tail assumption) is not formalized; the confidence of the ball event is a hypothesis of milestone 6.
  • A formalization in which the ball is taken only over distributions supported on the training samples, or in which the label term of the metric is dropped, trivializes the second constraint group of (7) and is ruled out: the ball here contains every Borel probability distribution on Ξ\XiΞ within the prescribed distance.
  • Infrastructure needed and reusable beyond this mission: type-1 optimal transport on product spaces with a label component, couplings and their marginals, and elementary properties of the logloss as a function of β\betaβ. Contributions of such supporting lemmas as independent theorems are welcome.

Selected references

  • S. Shafieezadeh-Abadeh, P. Mohajerin Esfahani, D. Kuhn, Distributionally Robust Logistic Regression, Advances in Neural Information Processing Systems 28 (NIPS 2015). https://arxiv.org/abs/1509.09259
  • P. Mohajerin Esfahani, D. Kuhn, Data-driven distributionally robust optimization using the Wasserstein metric: performance guarantees and tractable reformulations, Mathematical Programming 171 (2018). https://arxiv.org/abs/1505.05116
  • N. Fournier, A. Guillin, On the rate of convergence in Wasserstein distance of the empirical measure, Probability Theory and Related Fields 162 (2015). https://arxiv.org/abs/1312.2128
  • S. Shafieezadeh-Abadeh, D. Kuhn, P. Mohajerin Esfahani, Regularization via Mass Transportation, Journal of Machine Learning Research 20 (2019). https://arxiv.org/abs/1710.10016
9 thms2 active usersReviewed
🏆Completed
CombinatoricsLinear OptimizationOptimization·Captain: Shuze Chen

Disjunctive Programming VI: Extended Formulations for Perfectly Matchable Subgraph PolytopesTextbook

Motivation

Many polytopes that arise from combinatorial optimization problems have no small facet description in their natural variable space, yet become describable by a compact linear system once lifted to a higher-dimensional space of auxiliary variables and projected back down — Chapter 2's own extended formulation of the convex hull of a disjunctive set is one instance of this phenomenon. This chapter turns the idea around: rather than using projection to build a compact formulation, it uses projection to prove integrality of a formulation that is already compact but whose integrality is not obvious from any standard sufficient condition (total unimodularity, balancedness, etc.). The technique is illustrated on three closely related combinatorial polytopes built from perfectly matchable, assignable, and path-decomposable vertex subsets of a graph or digraph — each proved integral by lifting to an edge- or arc-variable space where total unimodularity is easy to check, then projecting.

Setting

For a finite vertex set VVV, the incidence vector of W⊆VW \subseteq VW⊆V is 111 on WWW, 000 elsewhere, and x(S):=∑i∈Sxix(S) := \sum_{i \in S} x_ix(S):=∑i∈S​xi​. A graph G(W)G(W)G(W) has a perfect matching if there is a fixed-point-free involution on WWW respecting adjacency. The PMS (Perfectly Matchable Subgraph) polytope of GGG is conv(X)\mathrm{conv}(X)conv(X) where XXX is the set of incidence vectors of such WWW; N(S):={j∉S:(i,j)∈E for some i∈S}N(S) := \{j \notin S : (i,j) \in E \text{ for some } i \in S\}N(S):={j∈/S:(i,j)∈E for some i∈S}.

For a digraph (V,A)(V,A)(V,A): G(W)G(W)G(W) is assignable if it admits a cycle decomposition (a permutation of WWW respecting arcs), giving the Assignable Subgraph Polytope. For an acyclic digraph with distinguished nodes s,ts,ts,t: G(W∪{s,t})G(W \cup \{s,t\})G(W∪{s,t}) admits an sss-ttt path decomposition if a collection of interior-node-disjoint sss-ttt paths covers it, giving the sss-ttt Path Decomposable Subgraph Polytope over W⊆V∖{s,t}W \subseteq V \setminus \{s,t\}W⊆V∖{s,t}. Γ(S)\Gamma(S)Γ(S) and Γ∗(S)\Gamma^*(S)Γ∗(S) are the corresponding out-neighborhood operators. For an arbitrary graph, c(S)c(S)c(S) counts the connected components of the induced subgraph G(S)G(S)G(S).

Formalization targets

Theorem 5.1 (goal) — the PMS polytope of a bipartite graph

0≤xi≤1 (i∈V),x(V1)−x(V2)=0,x(S)−x(N(S))≤0  (S⊆V1).0 \le x_i \le 1\ (i \in V), \qquad x(V_1) - x(V_2) = 0, \qquad x(S) - x(N(S)) \le 0\ \ (S \subseteq V_1).0≤xi​≤1 (i∈V),x(V1​)−x(V2​)=0,x(S)−x(N(S))≤0  (S⊆V1​).

Theorem 5.2 — the Assignable Subgraph Polytope

0≤xi≤1 (i∈V),x(S∖Γ(S))−x(Γ(S)∖S)≤0(S⊆V).0 \le x_i \le 1\ (i \in V), \qquad x(S \setminus \Gamma(S)) - x(\Gamma(S) \setminus S) \le 0 \quad (S \subseteq V).0≤xi​≤1 (i∈V),x(S∖Γ(S))−x(Γ(S)∖S)≤0(S⊆V).

Theorem 5.3 — the sss-ttt Path Decomposable Subgraph Polytope

0≤xi≤1 (i∈V),x(S∖Γ∗(S))−x(Γ∗(S)∖S)≤0(S⊆V∖{s,t}).0 \le x_i \le 1\ (i \in V), \qquad x(S \setminus \Gamma^*(S)) - x(\Gamma^*(S) \setminus S) \le 0 \quad (S \subseteq V \setminus \{s,t\}).0≤xi​≤1 (i∈V),x(S∖Γ∗(S))−x(Γ∗(S)∖S)≤0(S⊆V∖{s,t}).

Theorem 5.4 — the PMS polytope of an arbitrary graph

0≤xi≤1 (i∈V),x(S)−x(N(S))≤∣S∣−c(S)0 \le x_i \le 1\ (i \in V), \qquad x(S) - x(N(S)) \le |S| - c(S)0≤xi​≤1 (i∈V),x(S)−x(N(S))≤∣S∣−c(S)

for every SSS all of whose components are single nodes or nonbipartite with odd order — the weakest faithful statement, since dropping the side condition would assert the inequality for subsets it does not hold for.

Significance

The results themselves. Each theorem gives an explicit, checkable linear system defining a polytope that arises naturally from a combinatorial covering/decomposition property, turning "does G(W)G(W)G(W) have property XXX" into a linear-programming feasibility question. Theorem 5.1 is the one the book proves in full and the template for the other three: bipartite matching, digraph assignment, and acyclic-digraph path decomposition are structurally parallel problems (all reduce to checking a König–Hall-type combinatorial condition), and the same lift-and-project technique handles all three uniformly. Theorem 5.4 extends the idea to arbitrary (non-bipartite) graphs at the cost of a sharper right-hand side and a component-based side condition, connecting to Edmonds' classical matching-polytope theory while remaining a genuinely different object (a polytope of coverable vertex sets, not of matchings themselves).

Formalizing it. No object in this mission — the PMS, Assignable, or Path Decomposable Subgraph polytopes, or their defining neighbor operators — exists on the platform prior to this mission. The closest platform result, MetricTSP.pm_polytope_decomposition (Edmonds' perfect matching polytope theorem, in edge-variable space over a fixed vertex set requiring every vertex matched), is a genuinely different object from Theorem 5.4's PMS polytope (vertex-variable space, vertices may be left unmatched by design) and is not reused as a kind: reference item; it is noted here as related, not equivalent.

Difficulty

The natural first attempt tries to verify each polytope's integrality directly, by checking a known sufficient condition (total unimodularity, balancedness) on the displayed vertex-space system itself. This fails: the book states explicitly that (5.5)'s coefficient matrix is not totally unimodular, which is exactly why the lift-to-edge-variables step is necessary at all. The real content of each theorem is the two-part argument: (1) the lifted system in edge/arc variables is totally unimodular (checkable directly), so its polyhedron is integral; and (2) the vertex- space system is exactly the projection of the lifted one — a nontrivial fact requiring Chapter 2's projection machinery, not merely an unfolding of definitions. Theorem 5.4's extra difficulty, flagged explicitly in the text, is that its projection cone is not pointed, so the proof must work with a finite generating set rather than extreme rays, and it suffices to find a subset of generators producing every facet rather than a complete generating set — a genuinely harder argument the book itself outsources to a citation.

Formalization scope

Undirected graphs use Mathlib's SimpleGraph; digraphs use a bare relation A : V → V → Prop (not required symmetric or irreflexive, matching the book's unrestricted notion). Bipartition is recorded via part : V → Bool (decidable by construction) rather than two Set V halves, keeping the sums x(V_1), x(V_2) computable over Finsets throughout. IsAssignable uses Equiv.Perm on the vertex-set subtype, since a cycle decomposition is exactly a permutation. IsComponentOf and IsBipartiteOn (Theorem 5.4) are built directly from reachability and 2-colorability rather than Mathlib's induced-subgraph/ConnectedComponent API, matching the "maximal connected subset" reading of "component" the book's own prose intends.

IsPathDecomposable (Theorem 5.3) encodes "admits an sss-ttt path decomposition" via a degree-constrained arc set (every interior node has exactly one incoming and one outgoing chosen arc, none entering sss or leaving ttt, at least one leaving sss) rather than an explicit list of vertex-disjoint paths — provably equivalent by the standard fact that an acyclic arc set with this degree pattern always decomposes into such a path family, and considerably lighter to state and reason about than constructing Path objects directly.

A trivializing formalization is ruled out explicitly: every theorem keeps the fractional box constraint 0≤xi≤10 \le x_i \le 10≤xi​≤1 rather than the integral xi∈{0,1}x_i \in \{0,1\}xi​∈{0,1} (per BRIEF.md's own warning, dropping the relaxation collapses the claim to a restatement of the combinatorial definition), and Theorem 5.1 is stated only for bipartite graphs — never generalized to subsume Theorem 5.4's genuinely different inequality system and side condition.

Selected references

  • E. Balas, Disjunctive Programming, Springer, 2018. DOI: 10.1007/978-3-030-00148-3, Chapter 5, §5.2.
  • M. O. Ball, U. Derigs, An analysis of alternate strategies for implementing matching algorithms, Networks 13 (1983) (cited in the text as [13], the origin of Theorems 5.2 and 5.3).
  • W. R. Pulleyblank, J. Edmonds, Facets of 1-matching polyhedra, in Hypergraph Seminar, Springer Lecture Notes in Mathematics 411 (1974) — the origin of the perfectly matchable subgraph polytope literature (cited in the text as [34], the origin of Theorem 5.1).
  • L. Lovász, M. D. Plummer, Matching Theory, Elsevier, 1986 (cited in the text as [35], the origin of Theorem 5.4).
6 thms2 active usersReviewed
🏆Completed
Linear OptimizationOptimization·Captain: Shuze Chen

Disjunctive Programming II: The Convex Hull of a Disjunctive Set via Lifting and ProjectionTextbook

Motivation

Convexity is what makes optimization tractable: a linear program's feasible region is convex, and this single fact underwrites the simplex method, LP duality, and everything built on top of them. Integer and disjunctive programs have no such luck — their feasible regions are unions of polyhedra, and a union of convex sets is generally not convex. If the convex hull of such a union could always be described compactly, integer programming would reduce to linear programming: optimize the same linear objective over the hull instead of the union, and any optimal vertex of the hull is automatically integral. The obstacle has always been that the convex hull of a union of polyhedra in Rn\mathbb{R}^nRn, described directly by its facets in Rn\mathbb{R}^nRn, typically needs exponentially many inequalities.

Balas's Theorem 2.1, proved in the 1970s and presented here as Chapter 2 of Disjunctive Programming (Balas, Springer 2018), breaks this exponential barrier by changing where the description lives. Rather than writing down the hull's facets in Rn\mathbb{R}^nRn, Theorem 2.1 lifts the problem to a higher-dimensional space — one auxiliary copy of Rn\mathbb{R}^nRn per polyhedron in the union — where the hull becomes the projection of a single, explicitly given polyhedron whose size grows only linearly with the number of polyhedra. This "extended formulation" technique, born here, became one of the central tools of modern integer programming and combinatorial optimization: representing a hard polytope as the projection of an easy one in higher dimension underlies, for instance, the polynomial-size extended formulations known for many combinatorial polytopes.

Setting

Fix a finite index set QQQ. For h∈Qh \in Qh∈Q, let AhA_hAh​ be a real matrix and bhb_hbh​ a vector of matching row dimension, and set Ph:={x∈Rn:Ahx≥bh}P_h := \{x \in \mathbb{R}^n : A_h x \ge b_h\}Ph​:={x∈Rn:Ah​x≥bh​}. The union F:=⋃h∈QPhF := \bigcup_{h \in Q} P_hF:=⋃h∈Q​Ph​ is the disjunctive set. Write Q∗:={h∈Q:Ph≠∅}Q^* := \{h \in Q : P_h \ne \emptyset\}Q∗:={h∈Q:Ph​=∅} for the feasible disjuncts.

The recession cone of a nonempty polyhedron PhP_hPh​ is Ch:={y:Ahy≥0}C_h := \{y : A_h y \ge 0\}Ch​:={y:Ah​y≥0}: the set of directions along which one can travel indefinitely from any point of PhP_hPh​ while remaining in PhP_hPh​. For a subset M⊆QM \subseteq QM⊆Q and sets ShS_hSh​ (h∈Mh \in Mh∈M), the (finite) Minkowski sum ∑h∈MSh\sum_{h \in M} S_h∑h∈M​Sh​ is {x:x=∑h∈Myh for some yh∈Sh}\{x : x = \sum_{h \in M} y^h \text{ for some } y^h \in S_h\}{x:x=∑h∈M​yh for some yh∈Sh​}. The maximal indices Q∗∗⊆Q∗Q^{**} \subseteq Q^*Q∗∗⊆Q∗ are the feasible disjuncts whose polyhedron is not contained in any other feasible disjunct's polyhedron.

Given a set S⊆Rn×βS \subseteq \mathbb{R}^n \times \betaS⊆Rn×β, its projection onto xxx is Projx(S):={x:∃ y∈β, (x,y)∈S}\mathrm{Proj}_x(S) := \{x : \exists\, y \in \beta,\ (x,y) \in S\}Projx​(S):={x:∃y∈β, (x,y)∈S}.

Formalization targets

Theorem 2.1 (goal) — the convex hull of a disjunctive set

cl conv(F)=Projx(P),P:={(x,{yh}h∈Q∗,{y0h}h∈Q∗):x= ⁣ ⁣∑h∈Q∗ ⁣ ⁣yh, Ahyh−bhy0h≥0, y0h≥0,  ⁣ ⁣∑h∈Q∗ ⁣ ⁣y0h=1}.\mathrm{cl}\,\mathrm{conv}(F) = \mathrm{Proj}_x(P), \qquad P := \Big\{(x, \{y^h\}_{h \in Q^*}, \{y^h_0\}_{h \in Q^*}) : x = \!\!\sum_{h \in Q^*}\!\! y^h,\ A_h y^h - b_h y^h_0 \ge 0,\ y^h_0 \ge 0,\ \!\!\sum_{h \in Q^*}\!\! y^h_0 = 1 \Big\}.clconv(F)=Projx​(P),P:={(x,{yh}h∈Q∗​,{y0h​}h∈Q∗​):x=h∈Q∗∑​yh, Ah​yh−bh​y0h​≥0, y0h​≥0, h∈Q∗∑​y0h​=1}.

This is the weakest correct statement: it claims only that the closed convex hull equals the projection of this specific lifted polyhedron PPP, not any stronger uniqueness or minimality claim about lifted representations in general (that refinement is Theorem 2.1's own follow-up discussion, not part of the theorem itself).

Corollary 2.2 — the extreme-point correspondence

Extreme points of cl conv(F)\mathrm{cl}\,\mathrm{conv}(F)clconv(F) correspond bijectively to the extreme points of PPP that place all of their mass on a single disjunct's coordinates.

Theorem 2.3 — tightness of the lifted representation

PQ=P  ⟺  Ck⊆∑h∈Q∗Ch∀ k∈Q∖Q∗,P_Q = P \iff C_k \subseteq \sum_{h \in Q^*} C_h \quad \forall\, k \in Q \setminus Q^*,PQ​=P⟺Ck​⊆h∈Q∗∑​Ch​∀k∈Q∖Q∗,

where PQP_QPQ​ is the variant of PPP indexed by all of QQQ rather than only Q∗Q^*Q∗.

Theorem 2.4 — from the convex hull to the union itself

Under two recession-cone conditions on Q∗∗Q^{**}Q∗∗, restricting PQP_QPQ​'s y0hy^h_0y0h​ variables to {0,1}\{0,1\}{0,1} makes its xxx-projection recover FFF itself, not merely cl conv(F)\mathrm{cl}\,\mathrm{conv}(F)clconv(F).

Significance

The result itself. Theorem 2.1 is the founding extended-formulation result of integer programming: it shows that every union of finitely many polyhedra — hence every mixed-integer program's feasible region, once expressed in disjunctive normal form — has a lifted description of size linear in the number of disjuncts, in stark contrast to the union's own facet description, which is generally exponential. Corollary 2.2 shows this lifting is not merely an upper bound with extraneous points: its extreme points correspond exactly, one-to-one, with the extreme points of the object it represents. Theorems 2.3 and 2.4 sharpen the picture: 2.3 tells you exactly when you can avoid knowing in advance which disjuncts are nonempty, and 2.4 tells you exactly when the same family of lifted systems, restricted to integral y0hy^h_0y0h​, describes the union FFF exactly rather than only its convex hull — this is Jeroslow and Lowe's characterization of when a disjunctive set is representable as the feasible region of an integer program at all.

Formalizing it. No object in this mission — the disjunctive set FFF, its lifted polyhedron PPP, recession cones of a union's components, or the extreme-point correspondence between a polytope and its lift — exists on the platform prior to this mission or anywhere in Mathlib (substrate.md records zero LP/polyhedron modules in Mathlib as of this writing). This mission is a from-scratch formalization of the book's central construction, restating (rather than importing) the disjunctive-set vocabulary introduced by the companion IntroDuality mission, per the series' convention that a draft mission cannot import another draft mission's definitions.

Difficulty

The natural first attempt at Theorem 2.1 tries to prove the two inclusions cl conv(F)⊆Projx(P)\mathrm{cl}\,\mathrm{conv}(F) \subseteq \mathrm{Proj}_x(P)clconv(F)⊆Projx​(P) and Projx(P)⊆cl conv(F)\mathrm{Proj}_x(P) \subseteq \mathrm{cl}\,\mathrm{conv}(F)Projx​(P)⊆clconv(F) by a direct facet-by-facet or vertex-by-vertex argument in Rn\mathbb{R}^nRn — exactly the exponential-size approach the theorem exists to avoid. The book's own first proof instead works entirely with convex combinations: an arbitrary point of cl conv(F)\mathrm{cl}\,\mathrm{conv}(F)clconv(F) is a combination of at most ∣Q∗∣|Q^*|∣Q∗∣ points, one from each polyhedron in the union (Carathéodory-style), which converts directly into a point of PPP by splitting the combination's weight across the lifted coordinates — and conversely, a point of PPP decomposes, disjunct by disjunct, into a convex combination of that disjunct's own vertices and extreme rays. Neither direction ever needs to enumerate facets of cl conv(F)\mathrm{cl}\,\mathrm{conv}(F)clconv(F) in Rn\mathbb{R}^nRn. The second proof (via projection and the polar cone WWW of the lifted system) shows the projected inequalities coincide with exactly the valid-inequality characterization of Theorem 1.2 (disjunctive Farkas), which is a different, complementary way of seeing why no facet of cl conv(F)\mathrm{cl}\,\mathrm{conv}(F)clconv(F) is missed.

Formalization scope

All theorems are stated over a finite index set Q : Type* with [Fintype Q], matrices Matrix (Fin (m h)) (Fin n) ℝ with m : Q → ℕ allowed to depend on h, and vectors in Fin n → ℝ. DisjunctiveSet, FeasibleIndices, MaximalIndices, RecessionCone, and MinkowskiSumOver fix the chapter's vocabulary; ProjX, LiftedPolyhedron, and IntegerRestricted fix the lifted system and its variants. cl conv F is Mathlib's closure (convexHull ℝ ·); extreme points use Mathlib's Set.extremePoints.

LiftedPolyhedron ranges its auxiliary vectors {yh}\{y^h\}{yh}, {y0h}\{y^h_0\}{y0h​} over all of QQQ rather than only the index subset Qidx the book restricts to, forcing the components outside Qidx to zero. This is an equivalent, Finset/decidability-free encoding — appending zero terms changes neither the defining sums nor the constraints — documented as a convention, not a weakening, in MODERATION_NOTES.md; the same definition instantiates both the (2.1)(2.1)(2.1) system (Qidx = Q^*) and the (2.1)Q(2.1)_Q(2.1)Q​ variant (Qidx = Q) that Theorem 2.3 compares.

A trivializing formalization is ruled out explicitly: taking ∣Q∗∣=1|Q^*| = 1∣Q∗∣=1 collapses the lifted system to x=y1x = y^1x=y1, y01=1y^1_0 = 1y01​=1, a vacuous restatement of x∈P1x \in P_1x∈P1​ that proves nothing about unions. Every theorem here is stated for a generic finite Q, never specialized to a fixed small size. Contributions beyond this mission's statements would need genuine polyhedral machinery (vertex/extreme-ray decomposition of a polyhedron, Carathéodory's theorem for cones) that is itself absent from Mathlib and would be welcome as a separate, reusable definitions layer.

Selected references

  • E. Balas, Disjunctive Programming, Springer, 2018. DOI: 10.1007/978-3-030-00148-3, Chapter 2, §2.1.
  • E. Balas, Disjunctive programming: Properties of the convex hull of feasible points, Discrete Applied Mathematics 89 (1998), 3–44 (reprint of a 1974 MSRR, cited in the text as [6], the origin of Theorem 2.1).
  • M. Conforti, M. Di Summa, Y. Faenza, On the size of extended formulations for polytopes associated with unions of polyhedra, SIAM Journal on Discrete Mathematics, cited in the text as [59] — establishes the tightness (minimum additional-variable count) of Theorem 2.1's lifted representation.
  • R. G. Jeroslow, J. K. Lowe, Modelling with integer variables, Mathematical Programming Study 22 (1984), 167–184 (cited in the text as [86]; the characterization behind Theorem 2.4's significance).
6 thms2 active usersReviewed
🏆Completed
Linear OptimizationOptimization·Captain: Shuze Chen

Disjunctive Programming I: Intersection Cuts and Duality for Disjunctive ProgramsTextbook

Motivation

Linear programming duality is one of the load-bearing facts of optimization: every feasible linear program has a dual whose value matches the primal's, and this correspondence drives the simplex method's stopping criterion, sensitivity analysis, and most complexity results for polyhedral problems. Integer and mixed-integer programs have no such duality theorem in general — the feasible region of a mixed-integer program is not convex, and the entire apparatus of linear programming duality is built on convexity.

Disjunctive programming, introduced by Egon Balas in the early 1970s, closes part of this gap. A disjunctive set is a union of finitely many polyhedra rather than a single polyhedron — the natural convex-analytic shadow of the "either/or" logical structure that integer variables encode (an integer variable's feasible region is a finite union of half-open pieces, hence a disjunction of the linear constraints that pin it to each value). Balas's insight was that disjunctive programs — linear programs whose feasible region is such a union — admit a strong duality theorem of their own, generalizing the linear-programming case rather than replacing it. This mission formalizes that theorem (Theorem 1.5 of Balas, Disjunctive Programming, Springer 2018) together with the two results the same chapter builds around it: the founding construction of the field, the intersection cut (Theorem 1.1, circa 1970), and the disjunctive generalization of Farkas' Lemma (Theorem 1.2), which characterizes every valid inequality — hence every cutting plane — for a disjunctive set.

Setting

Fix a finite index set QQQ. For each h∈Qh \in Qh∈Q, let AhA_hAh​ be a real mh×nm_h \times nmh​×n matrix and bh∈Rmhb_h \in \mathbb{R}^{m_h}bh​∈Rmh​, and set Ph:={x∈Rn:Ahx≥bh}P_h := \{x \in \mathbb{R}^n : A_h x \ge b_h\}Ph​:={x∈Rn:Ah​x≥bh​}. The union F:=⋃h∈QPhF := \bigcup_{h \in Q} P_hF:=⋃h∈Q​Ph​ is a disjunctive set: any (linear) system of inequalities combined with the logical connectives "and", "or", "not" reduces, via its disjunctive normal form, to a set of exactly this shape. Because a union of convex sets need not be convex, FFF is generally nonconvex even though each PhP_hPh​ is a polyhedron.

A disjunctive program minimizes a linear objective over such a union:

(DP)z0=min⁡{cx:x∈⋃h∈QXh},Xh:={x:Ahx≥bh, x≥0}.(DP)\qquad z_0 = \min\Big\{ c x : x \in \textstyle\bigcup_{h \in Q} X_h \Big\}, \qquad X_h := \{x : A_h x \ge b_h,\ x \ge 0\}.(DP)z0​=min{cx:x∈⋃h∈Q​Xh​},Xh​:={x:Ah​x≥bh​, x≥0}.

Its dual (DD)(DD)(DD) pairs a scalar www with one dual multiplier vector uhu_huh​ per disjunct, requiring w≤uhbhw \le u_h b_hw≤uh​bh​ and uhAh≤cu_h A_h \le cuh​Ah​≤c, uh≥0u_h \ge 0uh​≥0, simultaneously for every h∈Qh \in Qh∈Q, and maximizes www. Write Q∗:={h∈Q:Xh≠∅}Q^* := \{h \in Q : X_h \ne \emptyset\}Q∗:={h∈Q:Xh​=∅} for the disjuncts whose primal system is feasible, and Q∗∗:={h∈Q:Uh≠∅}Q^{**} := \{h \in Q : U_h \ne \emptyset\}Q∗∗:={h∈Q:Uh​=∅} (with Uh:={uh≥0:uhAh≤c}U_h := \{u_h \ge 0 : u_h A_h \le c\}Uh​:={uh​≥0:uh​Ah​≤c}) for those whose dual system is feasible.

The theorems below also use two objects from the origin of the subject (§1.2): given a basic solution xˉ\bar xxˉ of a linear program's optimal simplex tableau, with basic index set III and nonbasic index set JJJ, the tableau's coefficients aˉij\bar a_{ij}aˉij​ (i∈Ii \in Ii∈I, j∈Jj \in Jj∈J) determine, for each nonbasic jjj, an extreme ray direction rjr^jrj of the associated LP cone. A convex set SSS is PIP_IPI​-free at xˉ\bar xxˉ if xˉ\bar xxˉ lies in the interior of SSS and that interior contains no point of the mixed-integer feasible set PIP_IPI​.

Formalization targets

Theorem 1.1 — the intersection cut

λj∗:=max⁡{λj≥0:xˉ+λjrj∈S},∑j∈J1λj∗ xj≥1.\lambda^*_j := \max\{\lambda_j \ge 0 : \bar x + \lambda_j r^j \in S\}, \qquad \sum_{j \in J} \frac{1}{\lambda^*_j}\, x_j \ge 1.λj∗​:=max{λj​≥0:xˉ+λj​rj∈S},j∈J∑​λj∗​1​xj​≥1.

The displayed inequality cuts off xˉ\bar xxˉ but excludes no point of PIP_IPI​, for any PIP_IPI​-free convex set SSS containing xˉ\bar xxˉ in its interior.

Theorem 1.2 — Farkas' Lemma for Disjunctive Sets

(∀x∈F, αx≥α0)  ⟺  (∀h∈Q∗, ∃ uh≥0, uhAh=α, α0≤uhbh).\big(\forall x \in F,\ \alpha x \ge \alpha_0\big) \iff \big(\forall h \in Q^*,\ \exists\, u_h \ge 0,\ u_h A_h = \alpha,\ \alpha_0 \le u_h b_h\big).(∀x∈F, αx≥α0​)⟺(∀h∈Q∗, ∃uh​≥0, uh​Ah​=α, α0​≤uh​bh​).

Theorem 1.5 (goal) — duality for disjunctive programs

Under the Regularity Condition — (Q∗≠∅(Q^* \ne \emptyset(Q∗=∅ and Q∖Q∗∗≠∅)⇒Q∗∖Q∗∗≠∅Q \setminus Q^{**} \ne \emptyset) \Rightarrow Q^* \setminus Q^{**} \ne \emptysetQ∖Q∗∗=∅)⇒Q∗∖Q∗∗=∅ — exactly one of:

  1. both (DP)(DP)(DP) and (DD)(DD)(DD) are feasible, each attains an optimum, and z0=w0z_0 = w_0z0​=w0​; or
  2. one of the two is infeasible, and the other is infeasible or has no finite optimum.

This is the weakest faithful statement of the theorem: it asserts only the shape of the dichotomy established by Balas, not any strengthened or specialized form of it.

Corollary 1.6 — necessity of the Regularity Condition

If the Regularity Condition fails, (DP)(DP)(DP) is feasible, and (DD)(DD)(DD) is infeasible, then (DP)(DP)(DP) still has a finite minimum — exhibiting the duality gap that opens up once the condition is dropped.

Significance

The results themselves. Theorem 1.5 is the mission-critical fact that makes disjunctive programming a genuine extension of linear programming rather than an unrelated combinatorial device: every LP-duality-based algorithmic tool (bounding, sensitivity, complementary-slackness optimality certificates) has a disjunctive-programming counterpart because of this theorem. Theorem 1.1's intersection cut is the historical seed of an entire branch of integer-programming algorithms — lift-and-project cuts, mixed-integer Gomory cuts, and the split closure (later missions of this series) all specialize or generalize it. Theorem 1.2 is the structural fact that makes cutting-plane generation for disjunctive sets tractable at all: every valid inequality decomposes into per-disjunct Farkas certificates.

Formalizing it. None of these results, nor the union-of-polyhedra machinery they are stated over, exist on the platform prior to this mission: the platform's existing Farkas' Lemma and linear-programming strong duality theorems (SmaleNinth.farkas_lemma, SmaleNinth.lp_strong_duality) are the ordinary single-polyhedron statements, which is exactly the special case ∣Q∣=1|Q|=1∣Q∣=1 of the theorems formalized here — genuinely different statements, not restatements. This mission is a from-scratch formalization of the disjunctive generalization, including the vocabulary (disjunctive sets, the paired primal/dual index sets Q∗,Q∗∗Q^*, Q^{**}Q∗,Q∗∗, the Regularity Condition) that the rest of the fifteen-mission Balas series builds on.

Difficulty

The obvious first attempt collapses the disjunctive dual (DD)(DD)(DD) to ∣Q∣|Q|∣Q∣ separate ordinary LP duals, one per disjunct, and tries to combine their individual strong-duality statements. This fails: (DD)(DD)(DD) couples all disjuncts through the single shared scalar www, which must simultaneously satisfy w≤uhbhw \le u_h b_hw≤uh​bh​ for every h∈Qh \in Qh∈Q at once, not disjunct-by-disjunct. The Regularity Condition exists precisely because this coupling can break down — Balas's own example (a two-term disjunctive program with an infeasible dual but a feasible, bounded primal) shows that without the condition, situation (2) of the dichotomy can fail: the primal can have a finite optimum with no matching dual optimum. Any formalization that omits the Regularity Condition, or weakens it to an informal restriction like "nondegenerate", either proves a false statement or proves nothing (a vacuous hypothesis), which Corollary 1.6 exists specifically to rule out.

Formalization scope

All three theorems are stated over a finite index set Q : Type* with [Fintype Q], real matrices Matrix (Fin (m h)) (Fin n) ℝ with row-dimension m : Q → ℕ allowed to depend on h (the book never assumes a common row count across disjuncts), and vectors in Fin n → ℝ. Poly, PolyNonneg, and DualPoly are the plain, nonnegative-orthant, and dual polyhedral systems respectively; FeasibleIndices and RegularityCondition pin Q∗Q^*Q∗/Q∗∗Q^{**}Q∗∗ and the Regularity Condition exactly as stated on p. 13. "No finite optimum" is formalized via UnboundedBelowOn / UnboundedAboveOn: nonempty (feasible) together with no finite bound on the objective, matching Balas's case (2), which explicitly distinguishes infeasibility from unboundedness.

A trivializing formalization is ruled out explicitly: fixing ∣Q∣=1|Q| = 1∣Q∣=1 collapses Theorem 1.5 to ordinary LP duality (already on the platform) and Theorem 1.2 to ordinary Farkas' Lemma, so both theorems are stated for a generic finite Q, never specialized. Theorem 1.5's "exactly one of" dichotomy is formalized as a logical Xor of the two situations, not a weaker Or, since the book asserts mutual exclusivity, not merely that one holds.

The intersection-cut theorem (1.1) is formalized over a generic finite index type ι standing for the full set of structural and surplus variables, with I J : Finset ι the basic/nonbasic partition; a complete development would additionally need the simplex-tableau apparatus connecting ι, I, J, and abar to an actual linear program, which lies outside this mission and belongs instead to the tableau-focused later missions of the series (SimplexTableau, RayCGLP). The extremeRay and PIFree definitions introduced here are local to this mission and are restated, not imported, by later missions that need related vocabulary — per the series' convention that a draft mission cannot import another draft mission's definitions.

Selected references

  • E. Balas, Disjunctive Programming, Springer, 2018. DOI: 10.1007/978-3-030-00148-3, Chapter 1.
  • E. Balas, Intersection cuts — a new type of cutting planes for integer programming, Operations Research 19 (1971), 19–39. (Theorem 1.1's origin, cited in the text as [4].)
  • E. Balas, Disjunctive programming, Annals of Discrete Mathematics 5 (1979), 3–51. (Cited in the text as [9], the origin of Theorem 1.5.)
8 thms2 active usersReviewed
🏆Completed
CombinatoricsConvex OptimizationGraph Theory·Captain: mikedeng1

Cones of Matrices and Set-Functions and 0–1 Optimization IV: Clique, Odd Hole, Odd Wheel and Odd Antihole Constraints Hold after One Round of N₊Research Paper

Motivation

The stable set problem (find a largest, or maximum-weight, set of pairwise non-adjacent nodes in a graph) is NP-hard, and its linear programming relaxations have been studied since the 1970s as a test bed for polyhedral combinatorics. Lovász and Schrijver (SIAM J. Optim. 1991) introduced a general lift-and-project procedure for 0–1 programs: lift a relaxation to a cone of (n+1)×(n+1)(n+1)\times(n+1)(n+1)×(n+1) matrices, impose conditions every 0–1 solution satisfies, and project back. Its semidefinite version, the operator N+N_+N+​, is one of the first systematic uses of positive semidefinite constraints in combinatorial optimization, and it is the ancestor of the Sherali–Adams, Lasserre and sum-of-squares hierarchies used today in approximation algorithms and proof complexity.

For the stable set problem the paper measures the strength of the operators by an index: how many rounds are needed before a given valid inequality is implied. This mission formalizes the paper's result that one round of N+N_+N+​ already implies four of the classical families of facets of the stable set polytope.

Timeline:

  • 1975: Chvátal shows that the rank constraint of a connected α-critical graph defines a facet of its stable set polytope (Chvátal 1975); clique, odd hole and odd antihole constraints are special rank constraints.
  • 1981–88: Grötschel, Lovász and Schrijver show that the weighted stable set problem is solvable in polynomial time for perfect and hhh-perfect graphs, through the theta body TH(G)\mathrm{TH}(G)TH(G) (Grötschel, Lovász, Schrijver 1988).
  • 1991: Lovász and Schrijver define the operators NNN and N+N_+N+​ and prove Corollary 2.15: clique, odd hole, odd wheel and odd antihole constraints have N+N_+N+​-index 1.

Setting

Vectors live in Rn+1\mathbb R^{n+1}Rn+1 with coordinates x0,x1,…,xnx_0, x_1, \dots, x_nx0​,x1​,…,xn​. The polar cone of KKK is K∗={u:uTx≥0 ∀x∈K}K^* = \{u : u^{\mathsf T}x \ge 0 \ \forall x \in K\}K∗={u:uTx≥0 ∀x∈K}. Let QQQ be the cone spanned by the 0–1 vectors with x0=1x_0 = 1x0​=1. For a convex cone K⊆QK \subseteq QK⊆Q, the matrix cone M+(K)M_+(K)M+​(K) consists of the symmetric positive semidefinite matrices Y=(yij)Y = (y_{ij})Y=(yij​) with yii=y0iy_{ii} = y_{0i}yii​=y0i​ for 1≤i≤n1 \le i \le n1≤i≤n and uTYv≥0u^{\mathsf T}Yv \ge 0uTYv≥0 for all u∈K∗u \in K^*u∈K∗, v∈Q∗v \in Q^*v∈Q∗. The operator is

N+(K)={Ye0:Y∈M+(K)},N_+(K) = \{Ye_0 : Y \in M_+(K)\},N+​(K)={Ye0​:Y∈M+​(K)},

and N+0(K)=KN_+^0(K) = KN+0​(K)=K, N+t(K)=N+(N+t−1(K))N_+^t(K) = N_+(N_+^{t-1}(K))N+t​(K)=N+​(N+t−1​(K)).

Let G=(V,E)G = (V, E)G=(V,E) be a finite graph with no isolated nodes (the paper's standing assumption for Section 2). STAB(G)\mathrm{STAB}(G)STAB(G) is the convex hull of incidence vectors χA\chi^AχA of stable sets AAA. FRAC(G)\mathrm{FRAC}(G)FRAC(G) is the polytope given by xi≥0x_i \ge 0xi​≥0 and xi+xj≤1x_i + x_j \le 1xi​+xj​≤1 for ij∈Eij \in Eij∈E. FR(G)⊆RV∪{0}\mathrm{FR}(G) \subseteq \mathbb R^{V\cup\{0\}}FR(G)⊆RV∪{0} is the cone xi≥0x_i \ge 0xi​≥0, xi+xj≤x0x_i + x_j \le x_0xi​+xj​≤x0​. The relaxations are

N+r(G)={x∈RV:(1,x)∈N+r(FR(G))},N_+^r(G) = \{x \in \mathbb R^V : (1, x) \in N_+^r(\mathrm{FR}(G))\},N+r​(G)={x∈RV:(1,x)∈N+r​(FR(G))},

so N+0(G)=FRAC(G)⊇N+1(G)⊇⋯⊇STAB(G)N_+^0(G) = \mathrm{FRAC}(G) \supseteq N_+^1(G) \supseteq \dots \supseteq \mathrm{STAB}(G)N+0​(G)=FRAC(G)⊇N+1​(G)⊇⋯⊇STAB(G). The N+N_+N+​-index of an inequality aTx≤ba^{\mathsf T}x \le baTx≤b valid for STAB(G)\mathrm{STAB}(G)STAB(G) is the least rrr with aTx≤ba^{\mathsf T}x \le baTx≤b valid for N+r(G)N_+^r(G)N+r​(G).

The four constraint families are:

  • clique: ∑i∈Bxi≤1\sum_{i\in B} x_i \le 1∑i∈B​xi​≤1 for a clique BBB;
  • odd hole: ∑i∈Cxi≤12(∣C∣−1)\sum_{i\in C} x_i \le \frac12(|C|-1)∑i∈C​xi​≤21​(∣C∣−1) for CCC inducing a chordless odd cycle;
  • odd wheel: ∑i∈U∖{u0}xi+∣U∣−22xu0≤∣U∣−22\sum_{i\in U\setminus\{u_0\}} x_i + \frac{|U|-2}{2}x_{u_0} \le \frac{|U|-2}{2}∑i∈U∖{u0​}​xi​+2∣U∣−2​xu0​​≤2∣U∣−2​ for UUU inducing an odd wheel with center u0u_0u0​ (an odd hole plus a node adjacent to all of it);
  • odd antihole: ∑i∈Dxi≤2\sum_{i\in D} x_i \le 2∑i∈D​xi​≤2 for DDD inducing a chordless odd cycle in the complement of GGG.

The contraction of a node vvv turns aTx≤ba^{\mathsf T}x \le baTx≤b into the inequality with the coefficients of vvv and its neighbours removed and right-hand side b−avb - a_vb−av​.

Formalization targets

Goal: Corollary 2.15

For every graph GGG without isolated nodes, each clique constraint (clique of size at least 3), odd hole constraint, odd wheel constraint and odd antihole constraint has N+N_+N+​-index exactly 1:

aTx≤b holds on N+1(G)and fails somewhere on FRAC(G).a^{\mathsf T}x \le b \text{ holds on } N_+^1(G) \quad\text{and fails somewhere on } \mathrm{FRAC}(G).aTx≤b holds on N+1​(G)and fails somewhere on FRAC(G).

Milestones

  1. Lemma 1.5: for a closed convex cone K⊆QK \subseteq QK⊆Q and aaa with ai≤0a_i \le 0ai​≤0 (i≥1i \ge 1i≥1), a0≥0a_0 \ge 0a0​≥0, if aTx≥0a^{\mathsf T}x \ge 0aTx≥0 holds on K∩GiK \cap G_iK∩Gi​ (where Gi={xi=x0}G_i = \{x_i = x_0\}Gi​={xi​=x0​}) for every iii with ai<0a_i < 0ai​<0, then it holds on N+(K)N_+(K)N+​(K).
  2. Lemma 2.14: if aTx≤ba^{\mathsf T}x \le baTx≤b is valid for STAB(G)\mathrm{STAB}(G)STAB(G), and the contraction of every node with positive coefficient is valid for N+r(G)N_+^r(G)N+r​(G), then aTx≤ba^{\mathsf T}x \le baTx≤b is valid for N+r+1(G)N_+^{r+1}(G)N+r+1​(G).
  3. Bipartite support (Section 2.c): an inequality valid for STAB(G)\mathrm{STAB}(G)STAB(G) whose nonzero-coefficient nodes induce a bipartite graph is valid for FRAC(G)\mathrm{FRAC}(G)FRAC(G).
  4. Contraction property (Section 2.d): contracting a node with positive coefficient in any of the four constraints leaves positive-coefficient nodes that induce a bipartite subgraph.

Further result

Corollary 2.19 (first sentence): the N+N_+N+​-index of a STAB(G)\mathrm{STAB}(G)STAB(G)-valid inequality aTx≤ba^{\mathsf T}x \le baTx≤b is at most the independence number of the subgraph induced by the nodes with positive coefficient.

Significance

Corollary 2.15 shows that a single round of N+N_+N+​, a relaxation over which one can optimize in polynomial time for each fixed number of rounds (the paper's Theorem 2.1), captures all clique, odd hole, odd wheel and odd antihole inequalities at once. Consequently N+(G)=STAB(G)N_+(G) = \mathrm{STAB}(G)N+​(G)=STAB(G) for every hhh-perfect graph, in particular for perfect and ttt-perfect graphs. The result is a standard reference point when comparing lift-and-project hierarchies, and the lemmas behind it (Lemma 1.5 and Lemma 2.14) are the paper's general tools for bounding N+N_+N+​-ranks.

The theorem was proved in 1991. To our knowledge it has not been machine-checked: this mission would produce the first formal development of the Lovász–Schrijver N+N_+N+​ operator, its iterates, and the stable set relaxations STAB\mathrm{STAB}STAB, FRAC\mathrm{FRAC}FRAC, FR\mathrm{FR}FR in Lean.

Difficulty

The lower bound (each constraint fails on FRAC(G)\mathrm{FRAC}(G)FRAC(G)) is a direct computation; the upper bound is where the work lies. The obvious approach, deriving each constraint from the linear conditions on the lifted matrix YYY alone, cannot succeed: those conditions define the linear operator NNN, and the goal is specifically about what positive semidefiniteness adds. The general lemmas are stated for arbitrary cones and require a working theory of polar cones and closedness in Rn+1\mathbb R^{n+1}Rn+1, including closedness of the iterates N+r(FR(G))N_+^r(\mathrm{FR}(G))N+r​(FR(G)), which the paper uses without comment. The graph-theoretic steps require facts about the stable set and fractional stable set polytopes of bipartite graphs and a careful case analysis of chordless odd cycles in a graph and in its complement, none of which is in Mathlib.

Formalization scope

  • Coordinates of Rn+1\mathbb R^{n+1}Rn+1 are indexed by Option ι, with none the special coordinate x0x_0x0​. For graphs, ι := V.
  • MMM is defined by condition (iii) with polar cones, not by its reformulations. Only M+M_+M+​, N+N_+N+​ and their iterates are defined; the linear operator NNN is not used.
  • Lemma 1.5 carries the hypothesis that KKK is closed. The paper takes it tacitly (all its cones are polyhedral); without it the lemma fails, since N+(K)N_+(K)N+​(K) depends only on the closure of KKK.
  • FR(G)\mathrm{FR}(G)FR(G) is defined by its constraints, which agree with the paper's "cone spanned by the vectors (1,x)(1,x)(1,x), x∈FRAC(G)x \in \mathrm{FRAC}(G)x∈FRAC(G)" because GGG has no isolated nodes. Every graph statement carries the no-isolated-nodes hypothesis.
  • Contraction is written on the same graph GGG as a zeroed coefficient vector, rather than on the subgraph G−Γ(v)−vG - \Gamma(v) - vG−Γ(v)−v.
  • Odd holes include triangles; odd antiholes have at least 5 nodes (a 3-node "antihole" is a stable set, for which the constraint is false); odd wheels are an odd hole plus a center adjacent to all its nodes.
  • Clique constraints in the goal are restricted to cliques with at least 3 nodes: cliques of size 1 or 2 give inequalities already valid on FRAC(G)\mathrm{FRAC}(G)FRAC(G), of index 0.
  • "N+N_+N+​-index at most rrr" is stated as validity on N+r(G)N_+^r(G)N+r​(G); the index itself is stated with IsLeast, never with an infimum that would default to 0 on an empty set.

A formalization asserting only validity on N+1(G)N_+^1(G)N+1​(G), or only for one fixed graph, would be weaker than the paper's statement and is ruled out: the goal states the exact index for all graphs without isolated nodes and all four families.

Not formalized: the linear operator NNN and its results, the polynomial-time separation results (Theorem 2.1, Corollaries 2.20–2.21), the theta-body results (Lemma 2.17, Corollary 2.18), graph indices (Corollary 2.16), and the second sentence of Corollary 2.19.

Reusable infrastructure includes the polar cone, the matrix cone M+M_+M+​ and the N+N_+N+​ operator (usable for any 0–1 program), the polytopes STAB\mathrm{STAB}STAB and FRAC\mathrm{FRAC}FRAC, and odd holes, antiholes and wheels as finite-set predicates. Contributions proving closedness of the iterates, the integrality of FRAC\mathrm{FRAC}FRAC for bipartite graphs, or the MMM-cone reformulations (iii′)–(iii″) are welcome.

Selected references

  • L. Lovász and A. Schrijver, Cones of matrices and set-functions and 0–1 optimization, SIAM Journal on Optimization 1(2), 1991, 166–190. https://doi.org/10.1137/0801013
  • M. Grötschel, L. Lovász and A. Schrijver, Geometric Algorithms and Combinatorial Optimization, Springer, 1988 (2nd ed. 1993). https://doi.org/10.1007/978-3-642-78240-4
  • V. Chvátal, On certain polytopes associated with graphs, Journal of Combinatorial Theory B 18, 1975, 138–154. https://doi.org/10.1016/0095-8956(75)90041-6
8 thms2 active usersReviewed
🏆Completed
CombinatoricsOptimizationTheoretical Computer Science·Captain: mikedeng1

Approximation Algorithms for Combinatorial Problems IV: Greedy Set Cover C1 Has Worst-Case Ratio H(k) on SC(k)Research Paper

Motivation

Set covering asks for the fewest members of a family of sets whose union is everything the family covers. It models crew scheduling, facility siting, test-suite reduction, logic minimization and fault testing; Johnson names the last two as its practical applications. Karp showed in 1972 that the decision version is NP-complete (Karp 1972), so in practice one runs a heuristic and asks how far from optimal it can be.

David S. Johnson's 1974 paper Approximation Algorithms for Combinatorial Problems (JCSS 9, 256–278) is one of the founding papers of the worst-case analysis of approximation algorithms. For set covering it analyses the obvious greedy rule, repeatedly take a set that covers the most still-uncovered points, and proves that on families whose sets have at most kkk elements its output is never more than the harmonic number H(k)=∑j=1k1/jH(k) = \sum_{j=1}^k 1/jH(k)=∑j=1k​1/j times the optimum, and that this factor is attained.

Timeline.

  • 1974: Johnson proves the H(k)H(k)H(k) bound for unweighted set cover with sets of size at most kkk, together with a matching family of examples (this mission).
  • 1975: Lovász proves the same bound for the fractional relaxation, giving an integrality-gap statement (Lovász 1975).
  • 1979: Chvátal extends the bound to weighted set cover, with the greedy rule choosing the set of least cost per newly covered point (Chvátal 1979).
  • 1998: Feige shows that no polynomial-time algorithm achieves (1−ε)ln⁡n(1-\varepsilon)\ln n(1−ε)lnn unless NP has slightly superpolynomial deterministic algorithms (Feige 1998), so the greedy guarantee is essentially the best possible.

Setting

An input FFF of SET COVERING I is a finite family {S1,…,Sp}\{S_1, \dots, S_p\}{S1​,…,Sp​} of finite sets. The set to be covered is T=⋃S∈FST = \bigcup_{S \in F} ST=⋃S∈F​S. A subcover is a subfamily F′⊆FF' \subseteq FF′⊆F with ⋃S∈F′S=T\bigcup_{S \in F'} S = T⋃S∈F′​S=T, and its measure is ∣F′∣|F'|∣F′∣. The optimum F∗F^*F∗ is the minimum measure of a subcover; FFF itself is a subcover, so the minimum exists. The subproblem SC(k) restricts the inputs to families no set of which has more than kkk elements.

Algorithm C1 keeps a family SUB of chosen sets, the set UNCOV of uncovered points, and an array SET[i][i][i] holding the still-uncovered part of SiS_iSi​. It starts with SUB =∅= \emptyset=∅, UNCOV =T= T=T, SET[i]=Si[i] = S_i[i]=Si​. While UNCOV is nonempty it chooses an index jjj with ∣SET[j]∣|\mathrm{SET}[j]|∣SET[j]∣ maximal, adds SjS_jSj​ to SUB, and removes SET[j][j][j] from UNCOV and from every SET[i][i][i]. When UNCOV is empty it returns SUB. When several indices tie at Step 3 any of them may be chosen, so one input can have several choosable outputs. Following Section 2 of the paper, the algorithm's value C1(F)C1(F)C1(F) is the worst choosable output, here the largest, and the ratio is r(C1,F)=C1(F)/F∗r(C1, F) = C1(F)/F^*r(C1,F)=C1(F)/F∗.

For the proof the paper introduces configurations K=⟨NK,UNCOVK,⟨SETK[1],…,SETK[NK]⟩⟩K = \langle N_K, \mathrm{UNCOV}_K, \langle \mathrm{SET}_K[1], \dots, \mathrm{SET}_K[N_K]\rangle\rangleK=⟨NK​,UNCOVK​,⟨SETK​[1],…,SETK​[NK​]⟩⟩ with ⋃iSETK[i]=UNCOVK\bigcup_i \mathrm{SET}_K[i] = \mathrm{UNCOV}_K⋃i​SETK​[i]=UNCOVK​, runs from a configuration (sequences of admissible choices ending when UNCOV is empty), Numbers(R)\mathrm{Numbers}(R)Numbers(R), the set of indices chosen in a run RRR, and calls a set MMM selectable from KKK if M=Numbers(R)M = \mathrm{Numbers}(R)M=Numbers(R) for some run RRR from KKK. Write n(K,i)=∣SETK[i]∣n(K, i) = |\mathrm{SET}_K[i]|n(K,i)=∣SETK​[i]∣.

Formalization targets

Goal: Theorem 4

For every k≥1k \ge 1k≥1:

for every input F∈SC(k) and every choosable F1:∣F1∣≤H(k)⋅F∗,\text{for every input } F \in SC(k) \text{ and every choosable } F_1:\quad |F_1| \le H(k)\cdot F^*,for every input F∈SC(k) and every choosable F1​:∣F1​∣≤H(k)⋅F∗, and some F∈SC(k) with F∗>0 has a choosable F1 with ∣F1∣=H(k)⋅F∗.\text{and some } F \in SC(k) \text{ with } F^* > 0 \text{ has a choosable } F_1 \text{ with } |F_1| = H(k)\cdot F^*.and some F∈SC(k) with F∗>0 has a choosable F1​ with ∣F1​∣=H(k)⋅F∗.

The paper states this as R[C1,SC(k)](n)≤∑j=1k(1/j)R[C1, SC(k)](n) \le \sum_{j=1}^k (1/j)R[C1,SC(k)](n)≤∑j=1k​(1/j) for all n>0n > 0n>0, with equality for all sufficiently large nnn. The two-part form above is the size-free equivalent.

Milestones

  1. Lemma 1. For a subcover F1F_1F1​ with index set M1={i:Si∈F1}M1 = \{i : S_i \in F_1\}M1={i:Si​∈F1​} and KKK the configuration after Step 1: F1F_1F1​ is choosable by C1 if and only if M1M1M1 is selectable from KKK.
  2. Lemma 2. For any configuration KKK, any M1M1M1 selectable from KKK and any M0M0M0 with ⋃i∈M0SETK[i]=UNCOVK\bigcup_{i \in M0} \mathrm{SET}_K[i] = \mathrm{UNCOV}_K⋃i∈M0​SETK​[i]=UNCOVK​:
∣M1∣≤∑i∈M0∑j=1n(K,i)1j.|M1| \le \sum_{i \in M0} \sum_{j=1}^{n(K,i)} \frac{1}{j}.∣M1∣≤i∈M0∑​j=1∑n(K,i)​j1​.
  1. Fig. 1. For every k≥1k \ge 1k≥1 there is an explicit input of SC(k)SC(k)SC(k) on k⋅k!k \cdot k!k⋅k! points with F∗=k!F^* = k!F∗=k! and a choosable output of k! H(k)k!\,H(k)k!H(k) sets.

Significance

The result. Theorem 4 is the first proof that greedy set cover has a worst-case guarantee depending only on the largest set size, and it pins the guarantee down exactly: the constant H(k)H(k)H(k) cannot be lowered for any kkk. Since H(k)≤1+ln⁡kH(k) \le 1 + \ln kH(k)≤1+lnk, it also gives the well-known 1+ln⁡n1 + \ln n1+lnn bound for general inputs. The H(k)H(k)H(k) bound and its later refinements are the standard reference point for analyses of greedy covering, dual fitting and submodular covering.

Formalizing it. The theorem has been proved since 1974. As far as a search of the platform shows, no machine-checked proof of it exists: the platform holds a Kearns–Vazirani-style statement ComputationalLearning.greedy_set_cover (the opt⋅ln⁡∣U∣\mathrm{opt}\cdot\ln|U|opt⋅ln∣U∣ form for a greedy sequence, still open) and a dual-fitting certificate lemma for weighted set cover, neither of which covers the SC(k)SC(k)SC(k) bound, the tie-breaking semantics or the tightness construction. A complete development provides both halves of Theorem 4, the configuration and run machinery of Lemmas 1–2, and the explicit Fig. 1 family.

Difficulty

The obvious argument charges each chosen set to the points it newly covers and compares the charges with an optimal cover. A statement about the initial input alone, with the original sizes of the optimal sets, does not survive a single greedy step: after a step the optimal sets are only partly uncovered and the remaining run faces a different instance. This is why Lemma 2 is stated for an arbitrary configuration, in terms of the current sizes n(K,i)n(K, i)n(K,i), and for an arbitrary covering subfamily M0M0M0. Because Step 3 breaks ties arbitrarily, the statement must hold for every admissible run, and a formalization that fixes one tie-breaking rule proves a weaker upper bound and cannot express the tightness example, which relies on adversarial ties at every stage.

For the tightness half, the difficulty is bookkeeping: showing that the k!/jk!/jk!/j blocks of each segment are admissible choices at each stage and that no cover uses fewer than k!k!k! sets.

Formalization scope

  • An input is an indexed family S : ι → Finset α over a finite index type ι and a ground type with decidable equality. The indices play the role of 1,…,N1, \dots, N1,…,N; two indices may carry the same set, which only widens the input class. The family, subcovers and F∗F^*F∗ are taken over the set of sets family S, as on the page. F∗F^*F∗ is a Finset.inf' over the nonempty finite set of subcovers; if T=∅T = \emptysetT=∅ then F∗=0F^* = 0F∗=0.
  • C1 is a nondeterministic step relation: a step is allowed for every index maximizing ∣SET[j]∣|\mathrm{SET}[j]|∣SET[j]∣. An output is choosable if a finite chain of steps from the initial state reaches a halting state with that SUB. No tie-breaking rule is fixed.
  • The paper's R[A,P](n)R[A, P](n)R[A,P](n) is a maximum over inputs of size at most nnn in an unspecified notation; it is replaced by the size-free two-part statement above, which is equivalent because RRR is a maximum over finitely many inputs and nondecreasing in nnn.
  • Ratios are stated multiplicatively in Q\mathbb{Q}Q (∣F1∣≤H(k)⋅F∗|F_1| \le H(k)\cdot F^*∣F1​∣≤H(k)⋅F∗), never as a quotient, so an input with F∗=0F^* = 0F∗=0 does not make the bound vacuous, and the attainment part requires F∗>0F^* > 0F∗>0. H(k)H(k)H(k) is Mathlib's harmonic k.
  • Configurations carry the covering condition as a field; runs are an inductive predicate on the list of chosen indices; Selectable K M means MMM is the set of indices of some run.
  • Lemma 1 assumes the family's sets are pairwise distinct (the paper's family is a set of sets); without that the index set {i:Si∈F1}\{i : S_i \in F_1\}{i:Si​∈F1​} may contain a duplicate index C1 never chose.
  • Trivializing formalizations are ruled out: a deterministic tie-break, a ratio written as a division, the original set sizes in place of n(K,i)n(K, i)n(K,i) in Lemma 2, or an attaining input with F∗=0F^* = 0F∗=0 would each change the theorem.

Contributions welcome: proofs of Lemma 2 (the core induction), of Lemma 1, of the Fig. 1 run, and of Theorem 4 from these; the configuration/run layer and the Fig. 1 family are reusable for other greedy covering analyses.

Selected references

  • David S. Johnson, Approximation algorithms for combinatorial problems, Journal of Computer and System Sciences 9 (1974), 256–278. https://doi.org/10.1016/S0022-0000(74)80044-9
  • Richard M. Karp, Reducibility among combinatorial problems, in Complexity of Computer Computations, Plenum, 1972, 85–103. https://doi.org/10.1007/978-1-4684-2001-2_9
  • László Lovász, On the ratio of optimal integral and fractional covers, Discrete Mathematics 13 (1975), 383–390. https://doi.org/10.1016/0012-365X(75)90058-8
  • Vašek Chvátal, A greedy heuristic for the set-covering problem, Mathematics of Operations Research 4 (1979), 233–235. https://doi.org/10.1287/moor.4.3.233
  • Uriel Feige, A threshold of ln n for approximating set cover, Journal of the ACM 45 (1998), 634–652. https://doi.org/10.1145/285055.285059
8 thms2 active usersReviewed
🏆Completed
CombinatoricsOptimizationTheoretical Computer Science·Captain: mikedeng1

Approximation Algorithms for Combinatorial Problems II: The Greedy Literal Algorithm B1 Has Worst-Case Ratio (k+1)/k on MS(k)Research Paper

Motivation

Maximum satisfiability asks for a truth assignment satisfying as many clauses of a propositional formula as possible. The paper notes that the restriction MS(k)MS(k)MS(k), in which every clause has at least kkk literals, is polynomial complete for every k≥1k \ge 1k≥1, so exact optimization is out of reach in general and one asks instead how close a fast algorithm is guaranteed to come. David S. Johnson's 1974 paper Approximation Algorithms for Combinatorial Problems (J. Comput. System Sci. 9, 256–278) set up a framework for exactly this question — optimization problems, nondeterministic approximation algorithms, and the worst-case ratio between the optimum and the algorithm's output — and applied it to subset-sum, maximum satisfiability, set covering, graph coloring and maximum clique. It is one of the founding papers of the theory of approximation algorithms.

Section 4 of the paper treats maximum satisfiability with two algorithms. This mission covers the first, a greedy literal-selection rule called B1, and its exact worst-case ratio (Theorem 2). A companion mission covers the weighted algorithm B2 (Theorem 3).

Timeline, for orientation:

  • 1971–1972: Cook and Karp establish NP-completeness of satisfiability and of many combinatorial problems.
  • 1974: Johnson proves that B1 has worst-case ratio exactly (k+1)/k(k+1)/k(k+1)/k on MS(k)MS(k)MS(k) and that the weighted algorithm B2 achieves 2k/(2k−1)2^k/(2^k-1)2k/(2k−1) (Theorems 2 and 3).
  • 1990s: semidefinite and LP-based algorithms (Goemans–Williamson, SIAM J. Discrete Math. 1994) improve the constants for general MAX-SAT.

Setting

Let L=⋃i>0{xi,xˉi}L = \bigcup_{i>0}\{x_i, \bar x_i\}L=⋃i>0​{xi​,xˉi​} be the set of literals; the complement of xix_ixi​ is xˉi\bar x_ixˉi​ and conversely. A clause is a finite set C⊆LC \subseteq LC⊆L. A truth assignment is a set T⊆LT \subseteq LT⊆L containing no complementary pair {xi,xˉi}\{x_i, \bar x_i\}{xi​,xˉi​}; it may leave variables unassigned. TTT satisfies CCC if C∩T≠∅C \cap T \ne \emptysetC∩T=∅.

An input is a finite set SSS of clauses. Its feasible solutions are the subsets S′⊆SS' \subseteq SS′⊆S satisfied by a single truth assignment, measured by ∣S′∣|S'|∣S′∣, and the optimum is

S∗=max⁡{∣S′∣:S′⊆S, some truth assignment satisfies every C∈S′}.S^* = \max\{|S'| : S' \subseteq S,\ \text{some truth assignment satisfies every } C \in S'\}.S∗=max{∣S′∣:S′⊆S, some truth assignment satisfies every C∈S′}.

The subproblem MS(k)MS(k)MS(k) admits only inputs whose clauses each contain at least kkk distinct literals.

Algorithm B1 keeps four variables: SUB (clauses already satisfied), LEFT (clauses not yet satisfied), TRUE (literals made true) and LIT (literals still available). It starts with SUB === TRUE =∅= \emptyset=∅, LEFT =S= S=S, LIT =L= L=L. While some literal of LIT occurs in a clause of LEFT, it picks a literal y∈y \iny∈ LIT contained in the most clauses of LEFT, moves those clauses YTYTYT from LEFT to SUB, adds yyy to TRUE, and removes yyy and yˉ\bar yyˉ​ from LIT. When no literal of LIT occurs in LEFT it returns SUB.

The choice of yyy is not determined when several literals tie. Following the paper's framework, every output reachable by some sequence of admissible choices is choosable, and the performance of B1 on SSS is the smallest ∣X∣|X|∣X∣ over choosable outputs XXX. The worst-case ratio on inputs of size at most nnn is

R[B1,MS(k)](n)=max⁡{S∗/B1(S):S∈MS(k), ∣S∣≤n}.R[B1, MS(k)](n) = \max\{S^*/B1(S) : S \in MS(k),\ |S| \le n\}.R[B1,MS(k)](n)=max{S∗/B1(S):S∈MS(k), ∣S∣≤n}.

Formalization targets

Goal: Theorem 2 (p. 262)

For all k≥1k \ge 1k≥1,

R[B1,MS(k)](n)≤k+1kfor all n>0,R[B1, MS(k)](n) \le \frac{k+1}{k}\quad\text{for all } n > 0,R[B1,MS(k)](n)≤kk+1​for all n>0,

with equality for all sufficiently large nnn. In the size-free form used here: every choosable output XXX on every S∈MS(k)S \in MS(k)S∈MS(k) satisfies k S∗≤(k+1) ∣X∣k\,S^* \le (k+1)\,|X|kS∗≤(k+1)∣X∣, and for every k≥1k \ge 1k≥1 some S∈MS(k)S \in MS(k)S∈MS(k) has a choosable XXX with ∣X∣>0|X| > 0∣X∣>0 and k S∗=(k+1) ∣X∣k\,S^* = (k+1)\,|X|kS∗=(k+1)∣X∣.

Milestones (from the proof of Theorem 2, pp. 262–263)

  1. In each iteration, the number of clauses saved (added to SUB) is at least the number of clauses remaining in LEFT that are wounded (lose a literal from LIT without being satisfied).
  2. When B1 halts, every clause left in LEFT is dead: each of its literals has had its complement made true.
  3. When B1 halts on an input of MS(k)MS(k)MS(k), ∣SUB∣≥k ∣LEFT∣|\mathrm{SUB}| \ge k\,|\mathrm{LEFT}|∣SUB∣≥k∣LEFT∣, and SUB and LEFT partition SSS.
  4. On the four-clause input {{x1,x2,x3},{xˉ1,x4,x5},{xˉ2,x6,x7},{xˉ3,x8,x9}}\{\{x_1,x_2,x_3\},\{\bar x_1,x_4,x_5\},\{\bar x_2,x_6,x_7\},\{\bar x_3,x_8,x_9\}\}{{x1​,x2​,x3​},{xˉ1​,x4​,x5​},{xˉ2​,x6​,x7​},{xˉ3​,x8​,x9​}} of MS(3)MS(3)MS(3), S∗=4S^* = 4S∗=4 while B1 may return three clauses.

Significance

The bound is stronger than a ratio: milestone 3 shows that B1 always satisfies at least kk+1∣S∣\tfrac{k}{k+1}|S|k+1k​∣S∣ clauses, whatever the optimum. The tightness half shows that this simple greedy rule cannot be analysed any better, which is what motivated the weighted algorithm B2 of the same section, with ratio 2k/(2k−1)2^k/(2^k-1)2k/(2k−1). The pair of theorems is an early instance of a now standard pattern: a potential-style counting argument for an upper bound, and an adversarial tie-breaking instance for the matching lower bound.

The result is proved in the paper; it has not, to our knowledge, been machine-checked. This mission produces a formal model of Johnson's framework for a maximization problem with a nondeterministic algorithm, a formal proof of the upper bound through the "saved versus wounded" accounting, and explicit tightness instances for every k≥1k \ge 1k≥1. The paper spells out only k=3k = 3k=3 and states that "similar examples can be constructed for any other k>0k > 0k>0"; the formal goal requires them for all kkk.

Difficulty

The upper bound needs an invariant over entire runs, not over a single step: a clause wounded in one iteration may be saved in a later one, so wounds and saves must be tallied globally, and the count of wounds received by a clause that ends in LEFT must be matched with its number of literals. That matching relies on the facts that B1 never makes both a literal and its complement true and that a clause containing a true literal has already left LEFT. Clauses containing both xix_ixi​ and xˉi\bar x_ixˉi​ are allowed and have to be handled.

The lower bound cannot be obtained from a fixed tie-breaking rule: the attaining run chooses negative literals whose count merely ties the maximum. For general kkk the instance has to be built so that every literal occurs in few enough clauses that the adversarial choice is admissible at every step; at k=1k = 1k=1 the paper's pattern degenerates and needs adjusting.

Formalization scope

  • A literal is a pair (variable index in N\mathbb NN, sign); a clause is a Finset of literals; an input is a Finset of clauses, so duplicate clauses are not allowed, as on the page. Tautological clauses are allowed.
  • A truth assignment is a Set of literals without a complementary pair (partial, as in the paper). S∗S^*S∗ is the maximum of ∣S′∣|S'|∣S′∣ over the finite nonempty family of satisfiable subsets, taken with Finset.sup'.
  • B1 is a nondeterministic run relation: a state holds SUB, LEFT, TRUE and the set of decided variables (LIT is its complement, since LLL is infinite); one step chooses any literal of LIT, of either sign, with maximum count; "choosable" is reachability of a halting state with the given SUB. No tie-break is fixed. A formalization that picks a variable and then its better sign, or that resolves ties deterministically, is a different algorithm and would make the tightness half false.
  • The ratio R[B1,MS(k)](n)R[B1, MS(k)](n)R[B1,MS(k)](n), whose problem size is left unspecified in the paper, is replaced by its size-free equivalent, and ratios are written multiplicatively in N\mathbb NN: k S∗≤(k+1)∣X∣k\,S^* \le (k+1)|X|kS∗≤(k+1)∣X∣. The tightness half requires ∣X∣>0|X| > 0∣X∣>0, so the empty input cannot witness it.
  • The running time O(nlog⁡n)O(n \log n)O(nlogn) is not stated.

Welcome contributions: proofs of the milestones, the invariants of reachable B1 states (SUB and LEFT partition SSS; TRUE is consistent and exactly covers the decided variables; no clause of LEFT meets TRUE), and the family of tightness instances for general kkk. The run-relation encoding of choosable outputs is reusable for the other algorithms of the paper.

Selected references

  • D. S. Johnson, Approximation algorithms for combinatorial problems, Journal of Computer and System Sciences 9 (1974), 256–278. https://doi.org/10.1016/S0022-0000(74)80044-9
  • R. M. Karp, Reducibility among combinatorial problems, in Complexity of Computer Computations, Plenum, 1972, 85–103. https://doi.org/10.1007/978-1-4684-2001-2_9
  • M. X. Goemans and D. P. Williamson, New 3/4-approximation algorithms for the maximum satisfiability problem, SIAM Journal on Discrete Mathematics 7 (1994), 656–666. https://doi.org/10.1137/S0895480192243516
8 thms2 active usersReviewed
🏆Completed
CombinatoricsOptimizationTheoretical Computer Science·Captain: mikedeng1

Approximation Algorithms for Combinatorial Problems I: The Subset-Sum Algorithms A_k Have Worst-Case Ratio (k+1)/kResearch Paper

Motivation

David S. Johnson's 1974 paper Approximation Algorithms for Combinatorial Problems (J. Comput. System Sci. 9 (1974) 256–278) is one of the founding papers of the theory of approximation algorithms. It asks, for optimization problems whose decision versions Karp had just shown to be polynomial complete, how close a fast heuristic can be guaranteed to come to the optimum in the worst case, and it measures this with a worst-case performance ratio that is still the standard yardstick.

Its first example is SUBSET-SUM, the simplest form of the knapsack problem: pack items of given sizes into a knapsack of capacity bbb so as to fill it as much as possible. For this problem the paper gives a family of algorithms AkA_kAk​, one for each k≥1k \ge 1k≥1, whose guaranteed ratio (k+1)/k(k+1)/k(k+1)/k tends to 111. It is one of the first examples of what is now called a polynomial-time approximation scheme: for every ϵ>0\epsilon > 0ϵ>0 there is a polynomial-time algorithm within a factor 1+ϵ1 + \epsilon1+ϵ of optimal. Sahni (1975) extended the idea to the knapsack problem with utilities, and Ibarra and Kim (1975) later obtained fully polynomial schemes for knapsack and subset-sum.

This mission formalizes Theorem 1 of the paper, the performance guarantee of AkA_kAk​ together with its tightness.

Setting

An input ⟨T,s,b⟩\langle T, s, b\rangle⟨T,s,b⟩ of SUBSET-SUM is a finite set TTT, a positive rational size s(x)s(x)s(x) for every x∈Tx \in Tx∈T, and a positive rational bound bbb. An approximate solution is a subset T′⊆TT' \subseteq TT′⊆T with m(T′)≤bm(T') \le bm(T′)≤b, where the measure is m(T′)=∑x∈T′s(x)m(T') = \sum_{x \in T'} s(x)m(T′)=∑x∈T′​s(x). The problem is a maximization problem with optimal measure

⟨T,s,b⟩∗=max⁡{ m(T′):T′⊆T, m(T′)≤b }.\langle T, s, b\rangle^* = \max\{\, m(T') : T' \subseteq T,\ m(T') \le b \,\}.⟨T,s,b⟩∗=max{m(T′):T′⊆T, m(T′)≤b}.

Fix k≥1k \ge 1k≥1 and call xxx big if s(x)>b/(k+1)s(x) > b/(k+1)s(x)>b/(k+1) and small otherwise. Algorithm AkA_kAk​ keeps a set SUB\mathrm{SUB}SUB, its measure SUM\mathrm{SUM}SUM, and the remaining elements LEFT\mathrm{LEFT}LEFT:

  1. SUB\mathrm{SUB}SUB is a subset of the big elements whose measure is as large as possible without exceeding bbb; SUM=m(SUB)\mathrm{SUM} = m(\mathrm{SUB})SUM=m(SUB) and LEFT=T∖SUB\mathrm{LEFT} = T \setminus \mathrm{SUB}LEFT=T∖SUB.
  2. If s(x)+SUM>bs(x) + \mathrm{SUM} > bs(x)+SUM>b for every x∈LEFTx \in \mathrm{LEFT}x∈LEFT, return SUB\mathrm{SUB}SUB.
  3. Otherwise pick y∈LEFTy \in \mathrm{LEFT}y∈LEFT with s(y)+SUMs(y) + \mathrm{SUM}s(y)+SUM as large as possible without exceeding bbb, move it from LEFT\mathrm{LEFT}LEFT to SUB\mathrm{SUB}SUB, add s(y)s(y)s(y) to SUM\mathrm{SUM}SUM, and return to step 2.

Steps 1 and 3 may have ties. Following the paper, a set T1T_1T1​ is choosable by AkA_kAk​ if some resolution of all ties produces it, and the performance Ak(u)A_k(u)Ak​(u) on input uuu is the smallest measure of a choosable output. The ratio is r(Ak,u)=u∗/Ak(u)≥1r(A_k, u) = u^*/A_k(u) \ge 1r(Ak​,u)=u∗/Ak​(u)≥1, and R[Ak](n)R[A_k](n)R[Ak​](n) is its maximum over inputs of size at most nnn.

Formalization targets

Goal: Theorem 1 (p. 260)

For k≥1k \ge 1k≥1 and n>0n > 0n>0,

R[Ak](n)≤k+1k,lim⁡n→∞R[Ak](n)=k+1k.R[A_k](n) \le \frac{k+1}{k}, \qquad \lim_{n \to \infty} R[A_k](n) = \frac{k+1}{k}.R[Ak​](n)≤kk+1​,n→∞lim​R[Ak​](n)=kk+1​.

Formally, for every k≥1k \ge 1k≥1: every choosable output T1T_1T1​ of every input satisfies k ⟨T,s,b⟩∗≤(k+1) m(T1)k\,\langle T,s,b\rangle^* \le (k+1)\,m(T_1)k⟨T,s,b⟩∗≤(k+1)m(T1​); and for every δ>0\delta > 0δ>0 some input has a choosable output T1T_1T1​ with m(T1)>0m(T_1) > 0m(T1​)>0 and ⟨T,s,b⟩∗>(k+1k−δ) m(T1)\langle T,s,b\rangle^* > \big(\tfrac{k+1}{k} - \delta\big)\,m(T_1)⟨T,s,b⟩∗>(kk+1​−δ)m(T1​).

Milestones

  1. For T1T_1T1​ choosable and T0T_0T0​ any approximate solution, m(T1BIG)≥m(T0BIG)m(T_1^{\mathrm{BIG}}) \ge m(T_0^{\mathrm{BIG}})m(T1BIG​)≥m(T0BIG​) (p. 260).
  2. If a small x∈Tx \in Tx∈T is not in a choosable T1T_1T1​, then s(x)+m(T1)>bs(x) + m(T_1) > bs(x)+m(T1​)>b, hence m(T1)>kb/(k+1)≥kk+1⟨T,s,b⟩∗m(T_1) > kb/(k+1) \ge \tfrac{k}{k+1}\langle T,s,b\rangle^*m(T1​)>kb/(k+1)≥k+1k​⟨T,s,b⟩∗ (p. 261).
  3. The stronger dichotomy: m(T1)=⟨T,s,b⟩∗m(T_1) = \langle T,s,b\rangle^*m(T1​)=⟨T,s,b⟩∗ or m(T1)≥kk+1 bm(T_1) \ge \tfrac{k}{k+1}\,bm(T1​)≥k+1k​b (p. 260).
  4. The lower-bound input T={a1,…,ak+2}T = \{a_1,\dots,a_{k+2}\}T={a1​,…,ak+2​}, s(a1)=1+εs(a_1) = 1+\varepsilons(a1​)=1+ε, s(ai)=1s(a_i) = 1s(ai​)=1 otherwise, b=k+1b = k+1b=k+1: its optimum is k+1k+1k+1, some output is choosable, and every choosable output has measure k+εk + \varepsilonk+ε (p. 261).

Significance

Theorem 1 shows that SUBSET-SUM admits polynomial-time algorithms with any worst-case ratio above 111, in contrast with the other problems of the paper (set covering, graph colouring, maximum clique), whose best known ratios grow with the input. The algorithms AkA_kAk​ are an early instance of the partial-enumeration schemes later used for knapsack-type problems. The tightness half shows that the analysis of AkA_kAk​ itself cannot be sharpened.

The theorem has a short published proof, but no machine-checked version is known; there is no subset-sum or knapsack approximation result on the platform. The mission produces a reusable model of SUBSET-SUM, a model of nondeterministic algorithms through a run relation that captures every tie-break, and a checked proof that the worst case is exactly (k+1)/k(k+1)/k(k+1)/k. The same modelling pattern (choosable outputs, worst-case ratio taken over them) is used in the sibling missions of this series for MAX-SAT, set covering and exact covering.

Difficulty

The arithmetic of the upper bound is short; the difficulty is in reasoning about the algorithm as a nondeterministic process. The natural first attempt, implementing AkA_kAk​ as a function with a fixed tie-breaking rule, proves a weaker statement: the guarantee must hold for every output the algorithm may return, including adversarial ties in step 1 (several maximum-measure sets of big elements) and step 3. Facts that are obvious for a single run, such as SUM\mathrm{SUM}SUM always equalling m(SUB)m(\mathrm{SUB})m(SUB) or which elements can enter SUB\mathrm{SUB}SUB after step 1, have to be established for the run relation as a whole. The lower bound requires tracing the run on the explicit input for general kkk: exactly k−1k-1k−1 unit elements are added after a1a_1a1​, and this must be shown for every choosable run, not only for one.

Formalization scope

  • Numbers. Sizes and the bound are rationals (ℚ), as in the paper; sizes are required to be positive on TTT and b>0b > 0b>0. The index kkk is a natural number with 1≤k1 \le k1≤k as a hypothesis; b/(k+1)b/(k+1)b/(k+1) is rational division, and "big" is the strict inequality s(x)>b/(k+1)s(x) > b/(k+1)s(x)>b/(k+1).
  • Optimum. opt u is Finset.sup' of the measure over the finite set of approximate solutions, which always contains ∅\emptyset∅; it is 000 when no element fits.
  • Run relation. Choosable k u T₁ states that some admissible step 1 choice, followed by a finite chain of admissible iterations (Relation.ReflTransGen), reaches a halting state returning T1T_1T1​. Every "closest to, without exceeding" is an existential choice among all maximizers.
  • Size-free restatement. The paper's input size ∣u∣|u|∣u∣ ("in some standard notation") is never fixed, so the goal quantifies over all inputs instead of over sizes. The upper bound for all choosable outputs is equivalent to R[Ak](n)≤(k+1)/kR[A_k](n) \le (k+1)/kR[Ak​](n)≤(k+1)/k for all nnn; since R[Ak]R[A_k]R[Ak​] is nondecreasing, the limit claim is equivalent to the supremum of the ratio over all inputs being (k+1)/k(k+1)/k(k+1)/k, which is the second part.
  • Multiplicative ratios. No ratio is written as a division, so an output of measure 000 cannot satisfy a bound vacuously; the lower-bound part requires m(T1)>0m(T_1) > 0m(T1​)>0. The value (k+1)/k(k+1)/k(k+1)/k is not claimed to be attained: the paper's family has ratio (k+1)/(k+ε)(k+1)/(k+\varepsilon)(k+1)/(k+ε).
  • Lower-bound input. A def on Fin (k + 2) exactly as on the page, with 0<ε<10 < \varepsilon < 10<ε<1 (the page leaves the range implicit; ε<1\varepsilon < 1ε<1 keeps a1a_1a1​ the only big element that fits when k=1k = 1k=1).
  • Ruled out. A formalization with a deterministic tie-break, with a bound of the form opt/m≤c\mathrm{opt}/m \le copt/m≤c in a field where x/0=0x/0 = 0x/0=0, or with tightness for a single fixed kkk would be trivial or weaker; none of these is the target.

Contributions welcome: proofs of the milestones and the goal, invariant lemmas for the run relation, and further sanity checks on small inputs. The running-time remark (O(nk)O(n^k)O(nk) for step 1) and Sahni's knapsack extension are not part of the mission.

Selected references

  • D. S. Johnson, Approximation algorithms for combinatorial problems, Journal of Computer and System Sciences 9 (1974) 256–278. https://doi.org/10.1016/S0022-0000(74)80044-9
  • S. Sahni, Approximate algorithms for the 0/1 knapsack problem, Journal of the ACM 22 (1975) 115–124. https://doi.org/10.1145/321864.321873
  • O. H. Ibarra, C. E. Kim, Fast approximation algorithms for the knapsack and sum of subset problems, Journal of the ACM 22 (1975) 463–468. https://doi.org/10.1145/321906.321909
  • R. M. Karp, Reducibility among combinatorial problems, in Complexity of Computer Computations, Plenum (1972) 85–103. https://doi.org/10.1007/978-1-4684-2001-2_9
8 thms2 active usersReviewed
🏆Completed
OptimizationTheoretical Computer Science·Captain: mikedeng1

An n Job, One Machine Sequencing Algorithm for Minimizing the Number of Late Jobs I: Moore's Algorithm Yields a Schedule with the Minimum Number of Late JobsResearch Paper

Motivation

A single machine must process a set of jobs, each with a processing time and a due-date, and a job that finishes after its due-date is late. Counting late jobs is the natural objective when a late order is simply lost, whatever its lateness. In the three-field notation of scheduling theory this is the problem 1 ∥ ∑Uj1\,\|\,\sum U_j1∥∑Uj​, and it is one of the few single-machine problems with a due-date objective that a simple greedy rule solves exactly.

J. Michael Moore gave that rule in 1968 (Management Science 15(1):102–109). The only exact method previously available was the Held–Karp dynamic program, which is exponential in the number of jobs. Moore's algorithm is two sorts plus at most n(n+1)/2n(n+1)/2n(n+1)/2 additions and comparisons. The rule, and the variant from the paper's Author's Supplement (credited to T. J. Hodgson and today called the Moore–Hodgson algorithm), is in every scheduling textbook, for example Brucker, Scheduling Algorithms, Ch. 4, and is the base case of later work on weighted and release-date variants.

Timeline:

  • 1955: J. R. Jackson shows that a job set can be scheduled with no late job if and only if the earliest-due-date order has none (Management Science Research Project report 43, UCLA).
  • 1968: Moore publishes the algorithm and its proof of optimality, with Hodgson's variant stated without proof.
  • 1970s onward: the weighted version 1 ∥ ∑wjUj1\,\|\,\sum w_jU_j1∥∑wj​Uj​ is shown NP-hard (Karp 1972, via knapsack), and 1 ∣ rj ∣ ∑Uj1\,|\,r_j\,|\,\sum U_j1∣rj​∣∑Uj​ likewise (Lenstra, Rinnooy Kan and Brucker 1977), so Moore's greedy rule does not extend to them.

Setting

A finite set JJJ of jobs is given. Job jjj has a processing time tj≥0t_j \ge 0tj​≥0 and a due-date DjD_jDj​, and the paper assumes tj≤Djt_j \le D_jtj​≤Dj​ for every job (a job that cannot finish on time even if started at time 000 is removed beforehand). The machine starts at time 000 and processes the jobs one after another, without idle time or preemption.

A schedule SSS of JJJ is an ordering (Ji1,…,Jin)(J_{i_1},\dots,J_{i_n})(Ji1​​,…,Jin​​) of all jobs of JJJ. The job in position kkk completes at Cik=ti1+⋯+tikC_{i_k} = t_{i_1} + \dots + t_{i_k}Cik​​=ti1​​+⋯+tik​​. The late set is L={Ji:Ci>Di}L = \{J_i : C_i > D_i\}L={Ji​:Ci​>Di​} and the early set is E={Ji:Ci≤Di}E = \{J_i : C_i \le D_i\}E={Ji​:Ci​≤Di​}. A schedule is optimal if no schedule of JJJ has fewer late jobs. AAA and RRR denote the early and late jobs of SSS, each kept in their order in SSS.

Moore's algorithm works on a current sequence and a list of rejected jobs.

  • Step 1: order the jobs by non-decreasing processing time (the shortest processing time rule).
  • Step 2: find the first late job JiqJ_{i_q}Jiq​​ of the current sequence. If there is none, stop.
  • Step 3: re-order Ji1,…,JiqJ_{i_1},\dots,J_{i_q}Ji1​​,…,Jiq​​ by non-decreasing due-date. If all of them are then early, keep the re-ordered sequence. Otherwise reject JiqJ_{i_q}Jiq​​ and remove it. Return to Step 2.

The output is the final current sequence sorted by due-dates, followed by the rejected jobs in any order.

In Lean, a schedule is IsSchedule J l, the late set is lateSet t D l, optimality is IsOptimal t D J l, AAA and RRR are earlyPart/latePart, and one pass of Steps 2–3 is the relation MooreStep t D, all in the namespace MooreLateJobs.NumLate.

Formalization targets

Goal: Moore's algorithm is optimal (The Algorithm, Step 2, p. 103)

Let l0l_0l0​ be a shortest-processing-time schedule of JJJ, and let a run of MooreStep from (l0,[ ])(l_0,[\,])(l0​,[]) reach a state (cur,rej)(\mathrm{cur},\mathrm{rej})(cur,rej) in which cur\mathrm{cur}cur has no late job. Then for every due-date ordering ADA_DAD​ of cur\mathrm{cur}cur and every ordering PPP of rej\mathrm{rej}rej,

(AD, P) is an optimal schedule for J.(A_D,\,P)\ \text{is an optimal schedule for } J.(AD​,P) is an optimal schedule for J.

All tie-breaks in both sorts are covered.

Milestones

In attack order:

  1. Lemma 1 (p. 105): every optimal schedule has the same number of late jobs as (A,R)(A,R)(A,R) and as every (A,P)(A,P)(A,P).
  2. Jackson's lemma (p. 105).
  3. Lemma 2 (p. 105): re-ordering AAA by due-dates keeps an optimal (A,R)(A,R)(A,R) schedule optimal.
  4. Lemma 3 (p. 105): a job that is late in some optimal schedule can be removed and appended.
  5. The repeated-elimination claim (p. 106): after removing jobs late in successive optimal schedules until the rest is feasible, (AD,P)(A_D,P)(AD​,P) is optimal.
  6. Cases 2) and 3) of the Selection Algorithm (p. 107): in either case the job JqJ_qJq​ is late in some optimal schedule.
  7. Progress and termination of the algorithm (p. 108).

A companion item states the p. 104 remark that the final current sequence need not be re-sorted: (cur,P)(\mathrm{cur},P)(cur,P) is already optimal.

Significance

The theorem shows that the minimum number of late jobs on one machine can be found in O(nlog⁡n)O(n\log n)O(nlogn) time, by a rule that also produces an optimal schedule of a very particular shape: due-date ordered early jobs first, then the late jobs in any order. Lemma 3's decomposition, that jobs late in some optimal schedule may be discarded one at a time, is the template reused for many related greedy results in scheduling.

The result is classical and fully proved on paper. To our knowledge no machine-checked proof of Moore's algorithm, of the Moore–Hodgson variant, or of Jackson's rule exists in Mathlib. This mission produces a checked proof of the algorithm as stated in the paper, with every tie-break allowed, together with reusable single-machine objects (schedules as lists, completion times, late sets) and Jackson's earliest-due-date feasibility lemma.

Difficulty

Neither ordering rule works alone. Sorting by due-dates alone gives a schedule with no late job whenever one exists, but it can make many jobs late once any must be. Keeping the shortest jobs first does not respect the due-dates at all. The step that fails in a direct greedy argument is the claim that the specific job JiqJ_{i_q}Jiq​​, the one just found late, belongs to the late set of some optimal schedule. That job is not in general the longest job of the prefix, and the paper has to treat separately the two cases in which it is rejected. On top of this, the algorithm re-sorts prefixes on the fly, so the claim has to be tied to the invariants of the run: the prefix is early and due-date sorted, and the jobs after it are at least as long as JiqJ_{i_q}Jiq​​.

Formalization scope

  • Jobs and times. Jobs form a type ι with decidable equality; JJJ is a Finset ι; t,D:ι→Rt, D : ι \to \mathbb{R}t,D:ι→R.
  • Standing hypotheses. Every statement that involves schedules assumes tj≥0t_j \ge 0tj​≥0 and tj≤Djt_j \le D_jtj​≤Dj​ on JJJ. The first is added: processing times are durations, and Jackson's lemma fails for negative times. The second is the paper's assumption on p. 102.
  • Schedules and completion times. A schedule is a duplicate-free list with exactly the jobs of JJJ. Positions are 0-based, and the job in position kkk completes at the sum of the first k+1k+1k+1 processing times. Lateness is strict (Cj>DjC_j > D_jCj​>Dj​).
  • Optimality compares against every schedule of the same job set.
  • Ties. Orderings "by due-dates" and "by processing times" are List.Pairwise with ≤. Ties are arbitrary, and every statement quantifies over all such orderings.
  • The algorithm. Steps 2–3 are the relation MooreStep. The re-ordered prefix is any due-date sorted permutation of the first q+1q+1q+1 jobs, and case 2) rejects the first late job JiqJ_{i_q}Jiq​​ itself, not the longest job of the prefix (that is Hodgson's variant). A run is Relation.ReflTransGen.

The goal must concern runs of this step relation from a shortest-processing-time schedule of JJJ. Replacing the run by an arbitrary set of rejected jobs satisfying invariants would state a different theorem. The goal is not vacuous: the progress and termination milestones show that a terminal state is always reached.

Contributions are welcome at every level. Useful ones include general lemmas on completion times under permutation and filtering of lists, a proof of Jackson's lemma, proofs of the Selection Algorithm cases, and a proof of Hodgson's variant.

Selected references

  • J. M. Moore, An n Job, One Machine Sequencing Algorithm for Minimizing the Number of Late Jobs, Management Science 15(1):102–109, 1968. https://doi.org/10.1287/mnsc.15.1.102
  • J. R. Jackson, Scheduling a Production Line to Minimize Maximum Tardiness, Research Report 43, Management Science Research Project, UCLA, 1955.
  • M. Held and R. M. Karp, A Dynamic Programming Approach to Sequencing Problems, J. SIAM 10(1):196–210, 1962. https://doi.org/10.1137/0110015
  • R. M. Karp, Reducibility among Combinatorial Problems, in Complexity of Computer Computations, 1972. https://doi.org/10.1007/978-1-4684-2001-2_9
  • J. K. Lenstra, A. H. G. Rinnooy Kan and P. Brucker, Complexity of Machine Scheduling Problems, Annals of Discrete Mathematics 1:343–362, 1977. https://doi.org/10.1016/S0167-5060(08)70743-X
  • P. Brucker, Scheduling Algorithms, 5th ed., Springer, 2007. https://doi.org/10.1007/978-3-540-69516-5
15 thms2 active usersReviewed
🏆Completed
CombinatoricsOptimization·Captain: mikedeng1

Scheduling with Deadlines and Loss Functions: On One Processor, Decreasing Penalty-to-Length Order Is Optimal When No Task Finishes Before Its DeadlineResearch Paper

Motivation

A processor, a machine shop or a single server must work through a set of jobs one at a time, and each job is costly when it is late. Deciding the order is the single-machine sequencing problem, the simplest and most studied model of scheduling theory. Robert McNaughton's 1959 article Scheduling with Deadlines and Loss Functions (Management Science 6(1):1–12) treats it for a computer that must run several tasks, each with a deadline and a loss that grows linearly with the lateness. Its §2 gives the first sufficient condition under which a simple ratio rule is optimal in the presence of deadlines, and shows that interrupting and resuming tasks ("splitting", now called preemption) never helps on one processor.

Timeline.

  • 1956: W. E. Smith, Various optimizers for single-stage production (Naval Research Logistics Quarterly 3), proves that sequencing jobs by non-increasing weight-to-processing-time ratio minimizes the total weighted completion time over non-preemptive sequences.
  • 1959: McNaughton, §2 of the present paper, proves independently that the same ratio order is optimal against all schedules, split or not and with idle time (Theorem 2.3), and extends it to deadlines when no task finishes early in that order (Theorem 2.4). §3 of the same paper gives the "wrap-around" rule for preemptive makespan on identical processors, and §4 the non-preemptive optimality for weighted completion time on several processors.
  • 1977: J. K. Lenstra, A. H. G. Rinnooy Kan and P. Brucker show that minimizing total weighted tardiness on one machine, the general problem of §2, is strongly NP-hard (Annals of Discrete Mathematics 1); this is why §2 gives a sufficient condition and not an algorithm.

Setting

There are mmm tasks (1),…,(m)(1),\dots,(m)(1),…,(m) for a single processor, and the present is time 000. Task (i)(i)(i) takes ai>0a_i > 0ai​>0 units of processing time, has a deadline did_idi​ and a penalty rate pi≥0p_i \ge 0pi​≥0. If (i)(i)(i) is finished at time Ci≤diC_i \le d_iCi​≤di​ there is no loss; otherwise the loss on (i)(i)(i) is pixp_i xpi​x, where x=Ci−dix = C_i - d_ix=Ci​−di​ is the time from the deadline to the completion. Thus the loss on a task completed at time ttt is

ℓi(t)=pimax⁡(0, t−di).\ell_i(t) = p_i \max(0,\ t - d_i).ℓi​(t)=pi​max(0, t−di​).

The ratio of task (i)(i)(i) is ri=pi/air_i = p_i / a_iri​=pi​/ai​.

A task may be split: part of it may run between times 4 and 6 and the remainder between times 8 and 11, and similarly in any finite number of parts. A schedule SSS is therefore a finite list of pieces, each a task together with a start and a stop time. It is feasible when every piece lies in [0,∞)[0,\infty)[0,∞) with start ≤\le≤ stop, no two pieces overlap in time, and the pieces of each task (i)(i)(i) have total length exactly aia_iai​. The completion time Ci(S)C_i(S)Ci​(S) is the latest stop time of a piece of (i)(i)(i), and the total loss is

c(S)=∑i=1mℓi(Ci(S)).c(S) = \sum_{i=1}^{m} \ell_i\bigl(C_i(S)\bigr).c(S)=i=1∑m​ℓi​(Ci​(S)).

For an order σ\sigmaσ of the tasks (σ(k)\sigma(k)σ(k) in position kkk), the sequenced schedule SσS_\sigmaSσ​ runs the tasks without splits and without unused time: σ(k)\sigma(k)σ(k) occupies [∑l<kaσ(l), ∑l≤kaσ(l)]\bigl[\sum_{l<k} a_{\sigma(l)},\ \sum_{l\le k} a_{\sigma(l)}\bigr][∑l<k​aσ(l)​, ∑l≤k​aσ(l)​]. The order is in decreasing rir_iri​ when k≤lk \le lk≤l implies rσ(l)≤rσ(k)r_{\sigma(l)} \le r_{\sigma(k)}rσ(l)​≤rσ(k)​. Finally c∗(S)c^*(S)c∗(S) denotes the total loss of SSS computed as if d1=⋯=dm=0d_1 = \dots = d_m = 0d1​=⋯=dm​=0.

Formalization targets

Goal: Theorem 2.4 (p. 5)

If σ\sigmaσ is in decreasing rir_iri​ and no task finishes before its deadline in SσS_\sigmaSσ​, i.e. di≤Ci(Sσ)d_i \le C_i(S_\sigma)di​≤Ci​(Sσ​) for every iii, then SσS_\sigmaSσ​ is feasible and

c(Sσ)≤c(S′)for every feasible schedule S′.c(S_\sigma) \le c(S') \qquad \text{for every feasible schedule } S'.c(Sσ​)≤c(S′)for every feasible schedule S′.

The competitors S′S'S′ may split tasks and leave the processor idle. The condition is sufficient but not necessary.

Milestones, in attack order

  1. Theorem 2.1 (p. 4): if both (i)(i)(i) and (j)(j)(j) run in the ai+aja_i + a_jai​+aj​ consecutive units of time after a time ttt past both deadlines and ri>rjr_i > r_jri​>rj​, their joint loss is strictly smaller when (i)(i)(i) goes first:
ℓi(t+ai)+ℓj(t+ai+aj)<ℓj(t+aj)+ℓi(t+aj+ai).\ell_i(t+a_i) + \ell_j(t+a_i+a_j) < \ell_j(t+a_j) + \ell_i(t+a_j+a_i).ℓi​(t+ai​)+ℓj​(t+ai​+aj​)<ℓj​(t+aj​)+ℓi​(t+aj​+ai​).
  1. The reduction in the proof of Theorem 2.2 (pp. 4–5): a feasible schedule with more than mmm pieces can be replaced by a feasible one with fewer pieces and no greater loss.
  2. Theorem 2.2 (p. 4): some optimal schedule, optimal among all feasible schedules, splits no task.
  3. Theorem 2.3 (p. 5): if d1=⋯=dm=0d_1 = \dots = d_m = 0d1​=⋯=dm​=0, the sequenced schedule in decreasing rir_iri​ minimizes the total loss over all feasible schedules.
  4. The display of the proof of Theorem 2.4 (p. 6): if no task finishes early in S=SσS = S_\sigmaS=Sσ​, then for every feasible S′S'S′,
c(S′)−c(S)≥c∗(S′)−c∗(S).c(S') - c(S) \ge c^*(S') - c^*(S).c(S′)−c(S)≥c∗(S′)−c∗(S).

Significance

The result. Theorem 2.3 is the ratio rule for total weighted completion time, in its strongest single-machine form: it holds against preemptive schedules and schedules with idle time, not only against permutations. Theorem 2.4 carries the rule over to deadlines and linear tardiness penalties under a checkable condition on one schedule. Since weighted tardiness is strongly NP-hard in general, a condition of this kind is what one can hope for, and the paper's two-step heuristic for general deadlines (p. 6) is built on it. Theorem 2.2, as the paper remarks (p. 6), "does not depend on the linear loss function": it makes non-preemptive scheduling without loss of generality for single-machine objectives of this kind.

Formalizing it. All results of §2 are proved in the paper and are textbook material; none has a machine-checked proof on the platform. The platform's Scheduling Algorithms V mission formalizes the multi-processor results of §§3–4 (via Brucker's textbook), and nothing there states a single-processor ratio rule with deadlines. This mission supplies a single-processor schedule model with splitting, the interchange lemma, the non-preemption theorem and the ratio rule, each over all feasible schedules.

Difficulty

The interchange argument of Theorem 2.1 compares only two schedules that differ in the order of two adjacent tasks. Turning it into optimality against every feasible schedule requires two further steps, and each fails if done naively. First, a competitor may split tasks and leave gaps; the interchange argument does not apply to such schedules, so a separate argument must remove splits without raising any completion time. Second, with deadlines the loss max⁡(0,t−di)\max(0, t - d_i)max(0,t−di​) is not linear in the completion time, so the ratio order is in general not optimal; the obvious attempt to repeat the interchange argument fails as soon as a task can finish before its deadline, since moving such a task later costs nothing. This is why Theorem 2.4 needs its hypothesis that no task finishes early, and why the paper leaves the general case to a heuristic.

Formalization scope

Tasks and positions are the zero-based indices of Fin m; times, lengths, deadlines and penalties are real numbers. A schedule is a List of pieces (task, start, stop), mirroring the public definition SchedulingAlgorithms_ParallelMachines with one processor. Feasibility requires 0≤0 \le0≤ start ≤\le≤ stop, pairwise disjoint pieces, and exact total length aia_iai​ per task; zero-length pieces and unsorted lists are allowed. The completion time is the maximum stop time of the task's pieces (000 for a task with no pieces, which feasibility excludes). "No split" means exactly one piece per task, so two abutting pieces count as a split. "Decreasing rir_iri​" is non-increasing, with ties in any order. "Minimal" and "optimal" are stated as ≤\le≤ against every feasible schedule, never as an infimum.

Standing assumptions, stated in every item: ai>0a_i > 0ai​>0 (tasks take time, and ri=pi/air_i = p_i/a_iri​=pi​/ai​ needs ai≠0a_i \ne 0ai​=0), and pi≥0p_i \ge 0pi​≥0 for Theorems 2.2–2.4 and the proof steps (penalties are non-negative; with a negative penalty and idle time allowed the loss is unbounded below). Theorem 2.1 carries no sign condition. No condition is placed on the deadlines.

A formalization that restricts the competitors of Theorems 2.2–2.4 to unsplit schedules, or to sequenced schedules of other orders, states a weaker theorem and is ruled out: every statement quantifies over all feasible schedules.

A complete development needs: sums over sublists of pieces, rearrangements of pieces of a schedule and their effect on completion times, and optimality over permutations of a finite set of tasks. The schedule model and the non-preemption argument are reusable for any single-machine regular objective. Contributions of intermediate lemmas on these points are welcome.

Selected references

  • R. McNaughton, Scheduling with Deadlines and Loss Functions, Management Science 6(1):1–12, 1959. https://doi.org/10.1287/mnsc.6.1.1
  • W. E. Smith, Various optimizers for single-stage production, Naval Research Logistics Quarterly 3(1–2):59–66, 1956. https://doi.org/10.1002/nav.3800030106
  • J. K. Lenstra, A. H. G. Rinnooy Kan, P. Brucker, Complexity of machine scheduling problems, Annals of Discrete Mathematics 1:343–362, 1977. https://doi.org/10.1016/S0167-5060(08)70743-X
  • P. Brucker, Scheduling Algorithms, 5th ed., Springer, 2007. https://doi.org/10.1007/978-3-540-69516-5
7 thms2 active usersReviewed
🏆Completed
Control TheoryConvex OptimizationOptimization·Captain: mikedeng1

Robust Solutions to Uncertain Semidefinite Programs II: An SDP Inner Approximation of the Robust Feasible Set under Structured PerturbationsResearch Paper

Motivation

A semidefinite program (SDP) minimizes a linear objective cTxc^TxcTx subject to a linear matrix inequality F(x)=F0+∑i=1mxiFi⪰0F(x) = F_0 + \sum_{i=1}^m x_i F_i \succeq 0F(x)=F0​+∑i=1m​xi​Fi​⪰0. In engineering applications the coefficient matrices are rarely known exactly: they come from measurements, from a model of a physical plant, or from a finite-precision implementation. El Ghaoui, Oustry and Lebret (SIAM J. Optim. 9(1), 1998) asked for solutions that remain feasible for every admissible value of the uncertain data, and showed how to compute such robust solutions by semidefinite programming. The paper appeared alongside Ben-Tal and Nemirovski's robust convex programming (Math. Oper. Res. 23(4), 1998) and is one of the two founding treatments of robust SDP.

When the uncertainty has structure (a block-diagonal perturbation, repeated scalar parameters, a symmetric matrix), the exact robust problem is NP-hard (El Ghaoui and Lebret, SIAM J. Matrix Anal. Appl. 18, 1997). This is the same obstacle that robust control meets in computing the structured singular value, and the remedy the paper uses, scaling matrices that commute with the perturbation structure, goes back to that literature (Doyle, IEE Proc. D 129, 1982; Fan, Tits and Doyle, IEEE Trans. Automat. Control 36, 1991). This mission formalizes the resulting tractable conservative approximation, Theorem 3.2 of the paper, together with the lemma it rests on and an application to integer feasibility problems.

Setting

Fix natural numbers m,n,p,qm, n, p, qm,n,p,q. The decision variable is x∈Rmx \in \mathbb{R}^mx∈Rm. The nominal data are affine maps

F(x)=F0+∑i=1mxiFi∈Rn×n,R(x)=R0+∑i=1mxiRi∈Rq×n,F(x) = F_0 + \sum_{i=1}^m x_i F_i \in \mathbb{R}^{n\times n}, \qquad R(x) = R_0 + \sum_{i=1}^m x_i R_i \in \mathbb{R}^{q\times n},F(x)=F0​+i=1∑m​xi​Fi​∈Rn×n,R(x)=R0​+i=1∑m​xi​Ri​∈Rq×n,

with every FiF_iFi​ symmetric, and fixed matrices L∈Rn×pL \in \mathbb{R}^{n\times p}L∈Rn×p, D∈Rq×pD \in \mathbb{R}^{q\times p}D∈Rq×p. A perturbation is a matrix Δ∈Rp×q\Delta \in \mathbb{R}^{p\times q}Δ∈Rp×q, and the perturbed constraint matrix is the linear-fractional representation (LFR)

F(x,Δ)=F(x)+LΔ(I−DΔ)−1R(x)+R(x)T(I−ΔTDT)−1ΔTLT,\mathbf{F}(x,\Delta) = F(x) + L\Delta(I - D\Delta)^{-1}R(x) + R(x)^T(I - \Delta^TD^T)^{-1}\Delta^TL^T,F(x,Δ)=F(x)+LΔ(I−DΔ)−1R(x)+R(x)T(I−ΔTDT)−1ΔTLT,

which is defined when det⁡(I−DΔ)≠0\det(I - D\Delta) \neq 0det(I−DΔ)=0. The perturbation ranges over a linear subspace D⊆Rp×q\mathcal{D} \subseteq \mathbb{R}^{p\times q}D⊆Rp×q, which encodes the structure, and is bounded by a level ρ>0\rho > 0ρ>0 in the spectral norm ∥Δ∥\|\Delta\|∥Δ∥ (the largest singular value). The robust feasible set is

Xρ={x:for every Δ∈D with ∥Δ∥≤ρ, det⁡(I−DΔ)≠0 and F(x,Δ)⪰0},\mathcal{X}_\rho = \{x : \text{for every } \Delta \in \mathcal{D} \text{ with } \|\Delta\| \le \rho,\ \det(I - D\Delta) \neq 0 \text{ and } \mathbf{F}(x,\Delta) \succeq 0\},Xρ​={x:for every Δ∈D with ∥Δ∥≤ρ, det(I−DΔ)=0 and F(x,Δ)⪰0},

and the robust SDP (RSDP) is to minimize cTxc^TxcTx over Xρ\mathcal{X}_\rhoXρ​.

The scaling set of D\mathcal{D}D is the linear subspace

B={(S,T,G)∈Rp×p×Rq×q×Rp×q:SΔ=ΔT, GΔT=−ΔGT for every Δ∈D}.\mathcal{B} = \{(S,T,G) \in \mathbb{R}^{p\times p}\times\mathbb{R}^{q\times q}\times\mathbb{R}^{p\times q} : S\Delta = \Delta T,\ G\Delta^T = -\Delta G^T \text{ for every } \Delta \in \mathcal{D}\}.B={(S,T,G)∈Rp×p×Rq×q×Rp×q:SΔ=ΔT, GΔT=−ΔGT for every Δ∈D}.

Formalization targets

Goal: Theorem 3.2 (p. 37), as an inclusion of feasible sets

For every xxx: if some (S,T,G)∈B(S,T,G) \in \mathcal{B}(S,T,G)∈B has S≻0S \succ 0S≻0, T≻0T \succ 0T≻0 and

[F(x)−LSLTR(x)T−LSDT+LGR(x)−DSLT+GTLTρ−2T−DSDT+DG+GTDT]≻0,\begin{bmatrix} F(x) - LSL^T & R(x)^T - LSD^T + LG \\ R(x) - DSL^T + G^TL^T & \rho^{-2}T - DSD^T + DG + G^TD^T\end{bmatrix} \succ 0,[F(x)−LSLTR(x)−DSLT+GTLT​R(x)T−LSDT+LGρ−2T−DSDT+DG+GTDT​]≻0,

then x∈Xρx \in \mathcal{X}_\rhox∈Xρ​, and in fact F(x,Δ)≻0\mathbf{F}(x,\Delta) \succ 0F(x,Δ)≻0 for every Δ∈D\Delta \in \mathcal{D}Δ∈D with ∥Δ∥≤ρ\|\Delta\| \le \rho∥Δ∥≤ρ. A companion item states the consequence for optimal values: the SDP value is an upper bound on the RSDP value, with both infima taken in the extended reals.

Milestones

  1. Lemma 3.2 (p. 37): the same implication for constant FFF, RRR and ρ=1\rho = 1ρ=1, with the matrix (13).
  2. The full-perturbation case (p. 37): for D=Rp×q\mathcal{D} = \mathbb{R}^{p\times q}D=Rp×q and p,q≥1p, q \ge 1p,q≥1, B\mathcal{B}B consists exactly of the triples (τIp,τIq,0)(\tau I_p, \tau I_q, 0)(τIp​,τIq​,0), with τ≥0\tau \ge 0τ≥0 when S⪰0S \succeq 0S⪰0.
  3. Theorem 5.6 (p. 48): if Fi=2LiRiF_i = 2L_iR_iFi​=2Li​Ri​ with ri=rank⁡Fir_i = \operatorname{rank} F_iri​=rankFi​, and xfeasx_{\mathrm{feas}}xfeas​ satisfies, for some λ≥0\lambda \ge 0λ≥0 and block-diagonal S=STS = S^TS=ST, G=−GTG = -G^TG=−GT,
[F(xfeas)−λI−LSLT12RT+LG12R−GLTS]≻0,\begin{bmatrix} F(x_{\mathrm{feas}}) - \lambda I - LSL^T & \tfrac12R^T + LG \\ \tfrac12R - GL^T & S\end{bmatrix} \succ 0,[F(xfeas​)−λI−LSLT21​R−GLT​21​RT+LGS​]≻0,

then every integer vector closest to xfeasx_{\mathrm{feas}}xfeas​ in the maximum norm satisfies F(z)⪰0F(z) \succeq 0F(z)⪰0.

Significance

The result. Theorem 3.2 replaces an NP-hard semi-infinite constraint, one matrix inequality for each admissible perturbation, by a single linear matrix inequality in the enlarged variable (x,S,T,G)(x, S, T, G)(x,S,T,G). Every point it certifies is robustly feasible, so its optimal value is a certified upper bound on the robust optimum and its optimizer is a usable robust solution. In the full case the scalings collapse to one multiplier τ\tauτ (milestone 2), which connects the bound to the exact reformulation of Section 3.1 of the paper. Theorem 5.6 shows the same machinery at work on a combinatorial problem: robustness against perturbations of size 1/21/21/2 in each coordinate of xxx turns an SDP-feasible point into an integer solution by rounding.

Formalizing it. The results are proved in the paper (Lemma 3.2 with the proof deferred to [16]); none of them has a machine-checked proof that this mission is aware of, and the platform has no linear-fractional or structured-perturbation results. The formalization also settles the exact form of the certificate: as printed, the matrix (13) and the LMI of Theorem 3.2 contain products that are dimensionally undefined, and this mission states the condition the proof actually yields (see the scope section).

Difficulty

The inequality to be proved is a statement about infinitely many perturbations, and F(x,Δ)\mathbf{F}(x,\Delta)F(x,Δ) depends on Δ\DeltaΔ through a matrix inverse. The natural first step, eliminating Δ\DeltaΔ by an exact S-procedure as in the full case, is not available: with a structured D\mathcal{D}D the set of pairs of vectors linked by some Δ∈D\Delta \in \mathcal{D}Δ∈D is not described by one quadratic inequality, and losslessness fails. The scalings in B\mathcal{B}B give several valid quadratic inequalities instead, and one must show that their combination controls every Δ\DeltaΔ in the norm ball, including the well-posedness claim det⁡(I−DΔ)≠0\det(I - D\Delta) \neq 0det(I−DΔ)=0, which is part of the conclusion rather than an assumption. The commutation condition SΔ=ΔTS\Delta = \Delta TSΔ=ΔT must be turned into an inequality for ∥Δ∥≤1\|\Delta\| \le 1∥Δ∥≤1, which requires more than the definition of the spectral norm. For Theorem 5.6 the block-diagonal perturbation family and the rescaling between ρ=1/2\rho = 1/2ρ=1/2 and the stated matrix must be matched to the general lemma.

Formalization scope

Matrices are Matrix (Fin a) (Fin b) ℝ; ≻0\succ 0≻0 and ⪰0\succeq 0⪰0 are Matrix.PosDef and Matrix.PosSemidef (both include symmetry); block matrices are Matrix.fromBlocks on Fin n ⊕ Fin q. The norm of a perturbation is the ℓ2\ell^2ℓ2 operator norm (open scoped Matrix.Norms.L2Operator), i.e. the largest singular value; the maximum norm in Theorem 5.6 is Mathlib's sup norm on Fin m → ℝ. D\mathcal{D}D is a Submodule. Affine maps are given by coefficient families indexed by Fin (m+1). Mathlib's matrix inverse is 000 at a singular matrix, so every statement pairs the LFR with det⁡(I−DΔ)≠0\det(I - D\Delta) \neq 0det(I−DΔ)=0. The standing assumption ρ>0\rho > 0ρ>0 of Section 3 is a hypothesis.

Readings and corrections of the printed statements:

  • (13) as printed is dimensionally inconsistent; we state the condition the proof yields, which coincides with the printed one when GGG is square and skew-symmetric and D\mathcal{D}D consists of symmetric matrices. Concretely, (11) prints G∈Rq×pG \in \mathbb{R}^{q\times p}G∈Rq×p with GΔ=−ΔTGTG\Delta = -\Delta^TG^TGΔ=−ΔTGT and (13) prints the blocks R−DSL−GLTR - DSL - GL^TR−DSL−GLT and T−GDT+DG−DSDTT - GD^T + DG - DSD^TT−GDT+DG−DSDT; the mission uses G∈Rp×qG \in \mathbb{R}^{p\times q}G∈Rp×q with GΔT=−ΔGTG\Delta^T = -\Delta G^TGΔT=−ΔGT and the blocks R−DSLT+GTLTR - DSL^T + G^TL^TR−DSLT+GTLT and T−DSDT+DG+GTDTT - DSD^T + DG + G^TD^TT−DSDT+DG+GTDT. The same correction applies to the LMI of Theorem 3.2 (with ρ−2T\rho^{-2}Tρ−2T). Theorem 5.6 is stated as printed.
  • "An upper bound on the RSDP (4) and a corresponding solution xxx can be computed by solving the SDP" is read as the inclusion of the SDP's feasible projection in Xρ\mathcal{X}_\rhoXρ​, for every xxx; the goal states it with the strict conclusion F(x,Δ)≻0\mathbf{F}(x,\Delta) \succ 0F(x,Δ)≻0 as well. The value form is a separate item.
  • In the full-perturbation remark, "for some τ≥0\tau \ge 0τ≥0" is stated under S⪰0S \succeq 0S⪰0, and "We then recover the exact results of section 3.1" is not formalized.
  • In Theorem 5.6, S\mathcal{S}S's index range "i=1,…,ni = 1,\dots,ni=1,…,n" is read as i=1,…,mi = 1,\dots,mi=1,…,m; the hypothesis ri=rank⁡Fir_i = \operatorname{rank}F_iri​=rankFi​ is kept.

Trivializing formalizations are ruled out: (0,0,0)∈B(0,0,0) \in \mathcal{B}(0,0,0)∈B always, so the hypotheses S≻0S \succ 0S≻0 and T≻0T \succ 0T≻0 are kept outside B\mathcal{B}B; D\mathcal{D}D is a subspace, not an arbitrary set; and the norm is the spectral norm, not Mathlib's default entrywise norm.

A complete development needs the square root of a positive definite matrix and its commutation with SSS and TTT, the spectral-norm characterization ΔΔT⪯∥Δ∥2I\Delta\Delta^T \preceq \|\Delta\|^2 IΔΔT⪯∥Δ∥2I, Schur-complement and congruence facts for block matrices, and a linear-fractional identity relating (I−DΔ)−1(I - D\Delta)^{-1}(I−DΔ)−1 to an auxiliary vector. These are reusable well beyond this mission; contributions of any of them, and of the value and rounding corollaries, are welcome.

Selected references

  • L. El Ghaoui, F. Oustry, H. Lebret, Robust Solutions to Uncertain Semidefinite Programs, SIAM J. Optim. 9(1):33–52, 1998. https://doi.org/10.1137/S1052623496305717
  • L. El Ghaoui, H. Lebret, Robust solutions to least-squares problems with uncertain data, SIAM J. Matrix Anal. Appl. 18:1035–1064, 1997. https://doi.org/10.1137/S0895479896298130
  • M. K. H. Fan, A. L. Tits, J. C. Doyle, Robustness in the presence of mixed parametric uncertainty and unmodeled dynamics, IEEE Trans. Automat. Control 36:25–38, 1991. https://doi.org/10.1109/9.62265
  • J. C. Doyle, Analysis of feedback systems with structured uncertainties, IEE Proc. D 129(6):242–250, 1982. https://doi.org/10.1049/ip-d.1982.0053
  • A. Ben-Tal, A. Nemirovski, Robust convex optimization, Math. Oper. Res. 23(4):769–805, 1998. https://doi.org/10.1287/moor.23.4.769
  • S. Boyd, L. El Ghaoui, E. Feron, V. Balakrishnan, Linear Matrix Inequalities in System and Control Theory, SIAM, 1994. https://doi.org/10.1137/1.9781611970777
5 thms2 active usersReviewed
🏆Completed
Convex OptimizationLinear OptimizationOptimization·Captain: mikedeng1

Validation of Subgradient Optimization I: The Core Problem Built from the Subgradient Iterates Solves the Dual Linear ProgramResearch Paper

Motivation

Subgradient optimization maximizes a concave function that is not differentiable by stepping along an arbitrary subgradient with a prescribed sequence of step sizes. It became a standard tool of integer programming after Held and Karp used it to compute the Lagrangian 1-tree bound for the traveling-salesman problem (Held & Karp 1971). Held, Wolfe and Crowder then tested it on the assignment problem, a traveling-salesman relaxation and a multicommodity flow problem (Held, Wolfe & Crowder 1974).

The method has one practical defect that the paper names at the start of its Section 6: it contains no test of optimality. The value w(πj)w(\pi^j)w(πj) approaches the maximum, but at no finite step does the method say that the maximum has been reached, or what the maximum is. Section 6 of the paper supplies such a test for the case where www is a minimum of finitely many affine functions. The finitely many subgradients produced by the iterates define a small linear program, the core problem, and from some iteration on this linear program already solves the full dual linear program. Its optimal value is therefore the exact maximum of www, obtained from quantities the method computes anyway. This is how the authors certified the optimal values reported in their experiments.

Timeline:

  • 1967–1969: Poljak proves that the subgradient iterates satisfy w(πj)→max⁡ww(\pi^j)\to\max ww(πj)→maxw when the step sizes tend to zero and have divergent sum (Poljak 1967; Poljak 1969).
  • 1971: Held and Karp apply the method to the 1-tree bound (Held & Karp 1971).
  • 1974: Held, Wolfe and Crowder prove that the core problem P(J,J∗)P(J,J^*)P(J,J∗) solves the dual linear program (Theorem 6.3) and give a sufficient condition for bounded iterates (Theorem 6.1).
  • 1996–1999: primal recovery from subgradient iterates is developed further, by convex combinations of the subgradients with weights derived from the step sizes (Sherali & Choi 1996; Larsson, Patriksson & Strömberg 1999).

Setting

Fix n≥0n\ge0n≥0 and write En=RnE^n=\mathbb R^nEn=Rn with the Euclidean inner product π⋅v\pi\cdot vπ⋅v. The data are K≥1K\ge1K≥1 scalars ckc_kck​ and vectors vk∈Env_k\in E^nvk​∈En, and

w(π)=min⁡{ck+π⋅vk:k=1,…,K}.(2.2)w(\pi)=\min\{c_k+\pi\cdot v_k : k=1,\dots,K\}.\qquad(2.2)w(π)=min{ck​+π⋅vk​:k=1,…,K}.(2.2)

The function www is assumed bounded above, the paper's standing assumption. An index kkk attains the minimum at π\piπ if ck+π⋅vk=w(π)c_k+\pi\cdot v_k=w(\pi)ck​+π⋅vk​=w(π).

A run of the subgradient algorithm consists of a starting point π0∈En\pi^0\in E^nπ0∈En, step sizes tj>0t_j>0tj​>0 and indices k(j)k(j)k(j) such that k(j)k(j)k(j) attains the minimum at πj\pi^jπj, and

πj+1=πj+tj vk(j)(j=0,1,… ).(2.6)\pi^{j+1}=\pi^j+t_j\,v_{k(j)}\qquad(j=0,1,\dots).\qquad(2.6)πj+1=πj+tj​vk(j)​(j=0,1,…).(2.6)

No rule for choosing among several minimizing indices is imposed. Write vj=vk(j)v^j=v_{k(j)}vj=vk(j)​ and cj=ck(j)c^j=c_{k(j)}cj=ck(j)​. The step-size conditions are

tj→0,∑j=0∞tj=∞.(2.7)t_j\to0,\qquad \sum_{j=0}^\infty t_j=\infty.\qquad(2.7)tj​→0,j=0∑∞​tj​=∞.(2.7)

The dual linear program of max⁡w\max wmaxw is

min⁡{∑kckyk:yk≥0, ∑kyk=1, ∑kykvk=0}.(6.1)\min\Big\{\sum_k c_ky_k : y_k\ge0,\ \sum_ky_k=1,\ \sum_ky_kv_k=0\Big\}.\qquad(6.1)min{k∑​ck​yk​:yk​≥0, k∑​yk​=1, k∑​yk​vk​=0}.(6.1)

For integers J<J∗J<J^*J<J∗ the core problem P(J,J∗)P(J,J^*)P(J,J∗) has one variable yjy_jyj​ for each iteration j∈[J,J∗]j\in[J,J^*]j∈[J,J∗]:

min⁡{∑j=JJ∗cjyj:yj≥0, ∑j=JJ∗yj=1, ∑j=JJ∗yjvj=0}.\min\Big\{\sum_{j=J}^{J^*}c^jy_j : y_j\ge0,\ \sum_{j=J}^{J^*}y_j=1,\ \sum_{j=J}^{J^*}y_jv^j=0\Big\}.min{j=J∑J∗​cjyj​:yj​≥0, j=J∑J∗​yj​=1, j=J∑J∗​yj​vj=0}.

An index chosen at several iterations contributes several identical columns. A point yyy of P(J,J∗)P(J,J^*)P(J,J∗) is sent to the point yˉk=∑{yj:J≤j≤J∗, k(j)=k}\bar y_k=\sum\{y_j : J\le j\le J^*,\ k(j)=k\}yˉ​k​=∑{yj​:J≤j≤J∗, k(j)=k} of (6.1). This aggregation preserves feasibility and objective value.

Formalization targets

Goal: Theorem 6.3 (p. 82)

Assume www is bounded above, (tj,πj,k(j))(t_j,\pi^j,k(j))(tj​,πj,k(j)) is a run satisfying (2.7), and {πj}\{\pi^j\}{πj} is bounded. Then

∀J ∃J∗>J:P(J,J∗) has a solution, and every solution of P(J,J∗) aggregates to a solution of (6.1).\forall J\ \exists J^*>J:\quad P(J,J^*)\text{ has a solution, and every solution of }P(J,J^*)\text{ aggregates to a solution of (6.1)}.∀J ∃J∗>J:P(J,J∗) has a solution, and every solution of P(J,J∗) aggregates to a solution of (6.1).

The goal states existence of J∗J^*J∗, which is what the paper claims. The paper's argument in fact gives the conclusion for every sufficiently large J∗J^*J∗. That stronger form is not the goal. Feasibility of P(J,J∗)P(J,J^*)P(J,J∗) (Lemma 6.2) or the inequality Value[P(J,J∗)]≥Value[(6.1)]\mathrm{Value}[P(J,J^*)]\ge\mathrm{Value}[(6.1)]Value[P(J,J∗)]≥Value[(6.1)], which holds for every feasible P(J,J∗)P(J,J^*)P(J,J∗), is not a formalization of the goal. The content is optimality in (6.1).

Milestones

  1. Eq. (2.10): if π∗\pi^*π∗ maximizes www and kkk attains the minimum at π\piπ, then w∗−w(π)≤vk⋅(π∗−π)w^*-w(\pi)\le v_k\cdot(\pi^*-\pi)w∗−w(π)≤vk​⋅(π∗−π).
  2. §6, p. 80 (display): under (2.6), (2.7) and www bounded above, lim⁡jw(πj)=max⁡w=w(π∗)\lim_j w(\pi^j)=\max w=w(\pi^*)limj​w(πj)=maxw=w(π∗) for some π∗\pi^*π∗. The iterates are not assumed bounded.
  3. Theorem 6.1: if every π≠0\pi\ne0π=0 has some π⋅vk<0\pi\cdot v_k<0π⋅vk​<0, every run satisfying (2.7) is bounded.
  4. Eq. (6.1): (6.1) has a solution, and its optimal value equals max⁡w\max wmaxw.
  5. Lemma 6.2: for any JJJ there is J∗>JJ^*>JJ∗>J with P(J,J∗)P(J,J^*)P(J,J∗) feasible, for bounded runs.

Significance

Theorem 6.3 turns an asymptotic method into one that returns an exact answer. Solving P(J,J∗)P(J,J^*)P(J,J∗) for growing J∗J^*J∗ produces a linear program of bounded size whose optimum is eventually the optimum of (6.1), and hence max⁡w\max wmaxw. In the Lagrangian applications, where (6.1) is the linear relaxation of a combinatorial problem, this yields both the bound and a primal solution of the relaxation. The theorem is the ancestor of the primal-recovery results listed in the timeline.

The mission produces a machine-checked version of the paper's Section 6, together with the input the paper takes on citation: Poljak's convergence theorem for divergent-series step sizes, specialized to piecewise-linear concave functions. Neither Poljak's theorem nor Theorem 6.3 is in Mathlib. The pieces are reusable: the convergence theorem applies to every Lagrangian dual solved by subgradient steps, and the duality between max⁡w\max wmaxw and (6.1) is linear-programming duality for a minimum of affine functions.

Difficulty

The inequality Value⁡P(J,J∗)≥Value⁡(6.1)\operatorname{Value}P(J,J^*)\ge\operatorname{Value}(6.1)ValueP(J,J∗)≥Value(6.1) is immediate, since aggregation maps feasible points to feasible points with the same objective. All of the content lies in the reverse inequality. That inequality ties a finite linear program to the limit of an infinite sequence, and it must hold for an arbitrary choice among tied minimizing indices. The iterates themselves need not converge, and under (2.7) the values w(πj)w(\pi^j)w(πj) are not monotone. So an argument that inspects a single iterate, or assumes that the method settles on one face of www, fails. The convergence statement of milestone 2 is not proved in the paper and is the heaviest single step. Feasibility of P(J,J∗)P(J,J^*)P(J,J∗) also needs its own argument, and it fails without the boundedness hypothesis.

Formalization scope

EnE^nEn is EuclideanSpace ℝ (Fin n), the index set is a finite nonempty type ι, and www is the finite minimum Finset.univ.inf'. A run is the predicate IsSubgradientRun c v t π k: positive steps, a minimizing index at every step, and update (2.6). It is not a function of π0\pi^0π0, so every tie-breaking rule is covered. (2.7) is StepSizeCond t: t → 0, and the partial sums tend to +∞+\infty+∞. Iterates are indexed from j=0j=0j=0. Boundedness is Bornology.IsBounded (Set.range π). The variables of P(J,J∗)P(J,J^*)P(J,J∗) are a function on N\mathbb NN of which only the values at J≤j≤J∗J\le j\le J^*J≤j≤J∗ enter. Optimality of yyy in either linear program means feasibility plus an objective no larger than that of every feasible point. Suprema are never taken over unbounded sets: every maximum of www is stated as attained at an explicit π∗\pi^*π∗.

A statement that only asserts feasibility of P(J,J∗)P(J,J^*)P(J,J∗), or only Value⁡P≥Value⁡(6.1)\operatorname{Value}P\ge\operatorname{Value}(6.1)ValueP≥Value(6.1), is not the theorem. The goal requires that the solutions of P(J,J∗)P(J,J^*)P(J,J∗) be optimal for (6.1).

Theorem 6.1 is printed for the step rule (2.8), but its proof uses w(πj)→w∗w(\pi^j)\to w^*w(πj)→w∗, the consequence of (2.7). The mission states it for (2.7), and its milestone title says so.

A complete development needs:

  • linear-programming duality for (6.1), including attainment;
  • the convergence theorem for divergent-series step sizes;
  • existence of a maximizer of a bounded-above minimum of finitely many affine functions;
  • basic facts on convex hulls of finitely many vectors in EnE^nEn.

The first three are reusable well beyond this mission. Contributions of any of them, as standalone theorems, are welcome.

Selected references

  • M. Held, P. Wolfe, H. P. Crowder, Validation of subgradient optimization, Mathematical Programming 6 (1974) 62–88. https://doi.org/10.1007/BF01580223
  • M. Held, R. M. Karp, The traveling-salesman problem and minimum spanning trees: Part II, Mathematical Programming 1 (1971) 6–25. https://doi.org/10.1007/BF01584070
  • B. T. Poljak, A general method of solving extremum problems, Soviet Mathematics Doklady 8 (1967) 593–597.
  • B. T. Poljak, Minimization of unsmooth functionals, USSR Computational Mathematics and Mathematical Physics 9 (1969) 14–29. https://doi.org/10.1016/0041-5553(69)90061-5
  • H. D. Sherali, G. Choi, Recovery of primal solutions when using subgradient optimization methods to solve Lagrangian duals of linear programs, Operations Research Letters 19 (1996) 105–113. https://doi.org/10.1016/0167-6377(96)00019-3
  • T. Larsson, M. Patriksson, A.-B. Strömberg, Ergodic, primal convergence in dual subgradient schemes for convex programming, Mathematical Programming 86 (1999) 283–312. https://doi.org/10.1007/s101070050090
7 thms2 active usersReviewed
PreviousPage 18 of 22Next

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paper
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me