On Minimizing a Convex Function Subject to Linear Inequalities II: Optimality Conditions for the Sum of the Largest Linear FormsResearch Paper
Motivation
In 1955 E. M. L. Beale showed how Dantzig's simplex method, which was built for linear objectives, can be carried over to certain nonlinear convex objectives that are minimized subject to linear inequalities (Beale 1955). Section 4 of that paper treats one such objective: the sum of the largest of a set of linear forms. Beale's motivation comes from the theory of games: "if the enemy has to choose out of a set of possible actions, and represents his average gain through using the th", then the defender wants to minimize the sum of the largest .
The same objective can be written as a linear program. One introduces a bound and requires every sum of forms to be at most . That formulation has constraints, which is unwieldy once and is large. Beale's alternative works with the nonlinear objective directly, and he needs a test that tells him when the current basic solution is already optimal. This mission formalizes that test, Theorem 1 of the paper.
The objective reappears in later work under other names: the sum of the largest components of a vector, the "top- sum", and times the conditional value-at-risk of an empirical distribution. Beale's paper is an early source for its optimality conditions.
Setting
There are real variables , indexed by in a finite set (possibly empty), and . Two linear forms in these variables are given,
together with further forms
For an integer the objective is
The sum of the largest of numbers is the largest total of any of them. Ties do not make it ambiguous.
The feasible region is fixed by a set of indices. The variables with and all the are free, and every other is restricted to . At the origin , all forms are equal to , so the origin is where fails to be differentiable. In Beale's algorithm the origin is the current basic solution: the measure how far the "borderline" forms sit from a chosen critical form, and collects the forms that are certainly among the largest.
Write and .
Formalization targets
Goal: Theorem 1 (a), p. 179
For , is minimized over the feasible region when all the and vanish if and only if
"Minimized" means a global minimum: at every feasible point.
Milestones
- Convexity (p. 179). is a convex function of for .
- Descent rules (second half of Theorem 1 (a), p. 179). When a condition of (4.5) fails, a stated move of one variable, or of all together, lowers below for every small enough step. There are six moves: if ; if and ; if ; if ; all if ; all if .
- The rearrangement identity (proof of Theorem 1 (a), p. 180). If , and , then
- Theorem 1 (b) (p. 180). For , the origin is a minimum if and only if (4.5) holds and for every . Otherwise some value of with the sign opposite to lowers .
Significance
Theorem 1 is the optimality test of Beale's simplex method for the sum-of-largest objective. The algorithm on pp. 178–179 changes nonbasic variables one at a time. When no single change is profitable it applies Theorem 1: either (4.5) holds and the current solution is optimal, or one of the six descent rules names the variable to change next. The test is exact even though the objective is not differentiable at the current point. It is a closed-form description of the subdifferential of a top- sum at a point where all the forms tie. The theorem is also the base case of the multi-group generalization that Beale mentions on p. 181.
The paper proves Theorem 1 by hand. To our knowledge neither the theorem nor the rearrangement identity behind it has been formalized in any proof assistant. The mission produces:
- a checked statement and proof of the test, including the degenerate cases and , which the paper does not discuss separately;
- the boundary case ;
- a reusable Lean definition of the sum of the largest entries of a finite real family, with its convexity.
Difficulty
Necessity, the "only if" direction, is the part the paper calls obvious: each descent rule changes linearly for small steps. Two features still have to be handled explicitly. The step must be small only in rule-dependent ways, and the ordering of the forms changes along the moves of rules 4 and 6.
Sufficiency is where the work lies. The naive argument, "the directional derivative in every coordinate direction is non-negative, so the origin is a minimum", fails because is not differentiable at the origin. Nonnegative derivatives along the coordinate axes do not control mixed directions in which several move by different amounts, which reorders the forms. Which forms are the largest then depends on the point, and the paper settles the configurations in which is among the largest by an informal appeal to the "essential symmetry" between and the other forms. A formal proof cannot leave that appeal informal: the forms are parametrised relative to (each is ), so the symmetry is a change of variables that has to be written down and shown to preserve (4.5).
Formalization scope
- Data. The variables are
z : Fin r → ℝ(anyr, including ) andu : Fin s → ℝ. The paper's for is Lean'su ffor . The coefficients form a structureForms r s. - Forms. The family is
Fin (s+1) → ℝ, with index for and indexf.succfor . The free set is aFinset (Fin r), and is a natural number cast to wherever it multiplies a coefficient. - Sum of the largest.
sumLargest τ vis the maximum over -element subsets of (Finset.sup'overpowersetCard). It is the junk for larger than the number of entries, a case no statement uses. - Minimality. "Minimized when all variables vanish" is the global statement for all with for . It is not a local minimum, and the sign constraints on restricted are kept: they are why the first condition of (4.5) is an inequality.
- Descent. " can be decreased by moving from zero" is a strict decrease for all step sizes in some interval , with every other variable at zero.
- No trivialization. The goal is an equivalence with no hypothesis beyond . Neither direction can be satisfied vacuously, and the cases and are included, as on the page.
- Added hypotheses. The rearrangement milestone assumes , because the paper's does not exist at . Its second line uses where the page misprints .
Needed infrastructure:
- basic lemmas on
sumLargest: its value at a constant family, at a family sorted by a monotone shift, and under adding a common constant; - the change of variables behind the paper's symmetry between and the other forms.
These lemmas are reusable for any top--sum or empirical-CVaR objective. Contributions are welcome at any level: lemmas about sumLargest, any of the milestones, or an alternative sufficiency proof through convexity and one-sided directional derivatives.
Not in scope: the pivoting rules (4.2)–(4.4), the degeneracy discussion on pp. 180–181, and the multi-group generalization, which the paper says is "cumbersome to state" and does not state.
Selected references
- E. M. L. Beale, On Minimizing a Convex Function Subject to Linear Inequalities, Journal of the Royal Statistical Society, Series B 17(2), 173–184, 1955. https://doi.org/10.1111/j.2517-6161.1955.tb00191.x
- G. B. Dantzig, A. Orden and P. Wolfe, The generalized simplex method for minimizing a linear form under linear inequality restraints, Pacific Journal of Mathematics 5(2), 183–195, 1955. https://doi.org/10.2140/pjm.1955.5.183
- R. T. Rockafellar and S. Uryasev, Optimization of conditional value-at-risk, Journal of Risk 2(3), 21–41, 2000. https://doi.org/10.21314/JOR.2000.038