Prove2Me
Navigate
DiscoverFormalpediaBlogsUsersMomentumMy Missions+
Prove2Me
⌕
Log in

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

Operations Research

889 missions · 467 completed

The discipline of applying mathematical analysis to complex decision problems in operations: allocating scarce resources, scheduling, routing, inventory, and the design of service and production systems. Drawing on mathematical programming, stochastic modeling, queueing, simulation, and game-theoretic reasoning, it seeks policies that perform provably well in systems shaped by constraints, congestion, and uncertainty.

Missions

Open422Completed467All889
🏆Completed
Linear OptimizationNumber TheoryOptimization·Captain: mikedeng1

An Application of Simultaneous Diophantine Approximation in Combinatorial Optimization: A Small Integral Objective with the Same Optimal Solutions and Dual BasesResearch Paper

Motivation

An algorithm for linear programming is strongly polynomial if the number of arithmetic operations it performs is bounded by a polynomial in the dimension of the problem alone (the number of variables and constraints), independently of the bit lengths of the numbers in the input. Many combinatorial optimization problems are linear programs over polyhedra of the form P={x∈Rn:Ax≤b}P = \{x \in \mathbb{R}^n : Ax \le b\}P={x∈Rn:Ax≤b} whose constraint matrix AAA has entries 0,+1,−10, +1, -10,+1,−1, but whose objective vector www is an arbitrary rational weight vector. Polynomial-time algorithms for such problems (for instance the ellipsoid-based algorithms of Grötschel, Lovász and Schrijver for maximum-weight cliques in perfect graphs, submodular flows, and matroid polyhedra) have running times that depend on the length of www.

Frank and Tardos (Combinatorica 1987) remove this dependence once and for all: they replace www by an integral objective w~\tilde ww~ whose entries have O(n3)O(n^3)O(n3) bits and which has exactly the same optimal solutions and the same optimal dual bases as www over every such polyhedron. Any algorithm that is polynomial in nnn and in the length of the objective then becomes strongly polynomial. The tool is simultaneous Diophantine approximation, used through the lattice-basis-reduction algorithm of Lenstra, Lenstra and Lovász (Math. Ann. 1982). The technique extends Tardos's strongly polynomial algorithm for linear programs with small constraint matrices (Oper. Res. 1986), which applies only to explicitly given programs.

Setting

For x∈Rnx \in \mathbb{R}^nx∈Rn write ∥x∥∞=max⁡j∣x(j)∣\|x\|_\infty = \max_j |x(j)|∥x∥∞​=maxj​∣x(j)∣ and ∥x∥1=∑j∣x(j)∣\|x\|_1 = \sum_j |x(j)|∥x∥1​=∑j​∣x(j)∣; sign⁡\operatorname{sign}sign takes the values −1,0,+1-1, 0, +1−1,0,+1.

Decomposition. Fix a positive integer NNN. A decomposition of w∈Rnw \in \mathbb{R}^nw∈Rn is an expression

w=∑i=1kλivi,λi>0, vi∈Zn.w = \sum_{i=1}^k \lambda_i v_i, \qquad \lambda_i > 0,\ v_i \in \mathbb{Z}^n.w=i=1∑k​λi​vi​,λi​>0, vi​∈Zn.

It satisfies condition (iii) if for i=2,…,ki = 2, \dots, ki=2,…,k the vector viv_ivi​ is nonzero and λi/λi−1≤1/(N∥vi∥∞)\lambda_i/\lambda_{i-1} \le 1/(N\|v_i\|_\infty)λi​/λi−1​≤1/(N∥vi​∥∞​): the coefficients decrease so quickly that each term is negligible against the previous one.

Preprocessing. Given a rational www and NNN, the paper's preprocessing algorithm finds a decomposition with k≤nk \le nk≤n, condition (iii), and the size bound (ii)' ∥vi∥∞≤2n2+nNn\|v_i\|_\infty \le 2^{n^2+n}N^n∥vi​∥∞​≤2n2+nNn, and outputs

w~=∑i=1kMk−ivi,M=2n2+nNn+1.\tilde w = \sum_{i=1}^k M^{k-i} v_i, \qquad M = 2^{n^2+n} N^{n+1}.w~=i=1∑k​Mk−ivi​,M=2n2+nNn+1.

Linear programs. Let AAA be an m×nm \times nm×n matrix with entries in {0,±1}\{0, \pm 1\}{0,±1} and b∈Rmb \in \mathbb{R}^mb∈Rm. The primal program is max⁡{wx:Ax≤b}\max\{wx : Ax \le b\}max{wx:Ax≤b} and the dual program is min⁡{yb:yA=w, y≥0}\min\{yb : yA = w,\ y \ge 0\}min{yb:yA=w, y≥0}. A point xˉ∈P\bar x \in Pxˉ∈P is www-maximal if wxˉ=max⁡(wx:x∈P)w\bar x = \max(wx : x \in P)wxˉ=max(wx:x∈P). A dual basis is a maximal set of row indices of AAA whose rows are linearly independent; it determines at most one yyy with yA=wyA = wyA=w supported on it (the basic dual solution), and it is an optimal dual basis if that yyy exists and is optimal for the dual program.

Formalization targets

Goal — Theorem 4.2 (p. 58)

For every w∈Qnw \in \mathbb{Q}^nw∈Qn, with N=(n+1)!+1N = (n+1)! + 1N=(n+1)!+1, there is w~∈Zn\tilde w \in \mathbb{Z}^nw~∈Zn with

∥w~∥∞≤24n3Nn(n+2)\|\tilde w\|_\infty \le 2^{4n^3} N^{n(n+2)}∥w~∥∞​≤24n3Nn(n+2)

such that for every 0,±10, \pm10,±1 matrix AAA with nnn columns and every bbb: (i) x∈Px \in Px∈P is www-maximal if and only if it is w~\tilde ww~-maximal; (ii) a set of rows of AAA is an optimal dual basis for www if and only if it is one for w~\tilde ww~. The vector w~\tilde ww~ depends on www only, not on AAA or bbb.

Milestones

  1. Dirichlet's theorem (p. 52): for N≥1N \ge 1N≥1 and α∈Rn\alpha \in \mathbb{R}^nα∈Rn there are p∈Znp \in \mathbb{Z}^np∈Zn and 1≤q≤Nn1 \le q \le N^n1≤q≤Nn with ∣qα(i)−p(i)∣<1/N|q\alpha(i) - p(i)| < 1/N∣qα(i)−p(i)∣<1/N for all iii.
  2. Theorem 3.1 (p. 53): every w∈Rnw \in \mathbb{R}^nw∈Rn has a decomposition with k≤nk \le nk≤n, ∥vi∥∞≤Nn\|v_i\|_\infty \le N^n∥vi​∥∞​≤Nn and condition (iii).
  3. Lemma 3.2 (pp. 54–55): under condition (iii), for integral bbb with ∥b∥1≤N−1\|b\|_1 \le N - 1∥b∥1​≤N−1, sign⁡(b⋅w)=sign⁡(b⋅vj)\operatorname{sign}(b \cdot w) = \operatorname{sign}(b \cdot v_j)sign(b⋅w)=sign(b⋅vj​) for the smallest jjj with b⋅vj≠0b \cdot v_j \ne 0b⋅vj​=0, and b⋅w=0b \cdot w = 0b⋅w=0 if there is no such jjj.
  4. Theorem 3.3 (p. 56): the preprocessed w~\tilde ww~ satisfies ∥w~∥∞≤24n3Nn(n+2)\|\tilde w\|_\infty \le 2^{4n^3}N^{n(n+2)}∥w~∥∞​≤24n3Nn(n+2) and sign⁡(w⋅b)=sign⁡(w~⋅b)\operatorname{sign}(w \cdot b) = \operatorname{sign}(\tilde w \cdot b)sign(w⋅b)=sign(w~⋅b) for all integral bbb with ∥b∥1≤N−1\|b\|_1 \le N-1∥b∥1​≤N−1.
  5. The case N=n+1N = n+1N=n+1 (p. 55): an integral w~\tilde ww~ with ∥w~∥∞≤24n3(n+1)n(n+2)\|\tilde w\|_\infty \le 2^{4n^3}(n+1)^{n(n+2)}∥w~∥∞​≤24n3(n+1)n(n+2) and w~(X)≤w~(Y)  ⟺  w(X)≤w(Y)\tilde w(X) \le \tilde w(Y) \iff w(X) \le w(Y)w~(X)≤w~(Y)⟺w(X)≤w(Y) for all subsets X,YX, YX,Y of coordinates.
  6. Lemma 4.1 (i) (p. 57): if sign⁡(w′⋅h)=sign⁡(w′′⋅h)\operatorname{sign}(w' \cdot h) = \operatorname{sign}(w'' \cdot h)sign(w′⋅h)=sign(w′′⋅h) for all integral hhh with ∥h∥1≤(n+1)!\|h\|_1 \le (n+1)!∥h∥1​≤(n+1)!, then w′w'w′ and w′′w''w′′ have the same maximizers over {Ax≤b}\{Ax \le b\}{Ax≤b} for every 0,±10, \pm10,±1 matrix AAA.
  7. Lemma 4.1 (ii) (p. 57): under the same hypothesis, a dual basis is optimal for w′w'w′ if and only if it is optimal for w′′w''w′′.

Significance

The result gives a general reduction: whenever a class of polyhedra with 0,±10, \pm10,±1 constraint matrices admits an optimization algorithm that is polynomial in nnn and in the length of the objective, it admits a strongly polynomial one. The paper applies this to maximum-weight cliques in perfect graphs, optimization over submodular flow polyhedra, and matroid polyhedra membership, and its Section 5 applies the same rounding to the integer programming algorithms of Lenstra and Kannan. The subset-sum corollary (milestone 5) is independently useful: every rational weight function on a finite set can be replaced by an integral one with O(n3)O(n^3)O(n3)-bit entries that orders all subset sums identically.

All statements of this mission have been proved on paper since 1987. None is formalized on Prove2Me, and Mathlib contains only the one-dimensional Dirichlet approximation theorem. The mission produces a machine-checked version of the exact statements, with the explicit constants of the paper; the complexity claims (operation counts, strong polynomiality) are not part of it.

Difficulty

The goal combines two independent parts. The number-theoretic part (milestones 1–5) needs a multidimensional Dirichlet theorem, an induction producing the decomposition, and exact inequality chains with the constants 2n2+nNn2^{n^2+n}N^n2n2+nNn and 24n3Nn(n+2)2^{4n^3}N^{n(n+2)}24n3Nn(n+2). The linear-programming part (milestones 6–7) needs bounds on the entries of inverses of nonsingular 0,±10, \pm10,±1 submatrices, the existence of optimal dual solutions supported on a dual basis, LP duality and complementary slackness. The obvious first idea, scaling www to an integer vector by a common denominator, preserves every sign but gives no bound on ∥w~∥∞\|\tilde w\|_\infty∥w~∥∞​ in terms of nnn; the bound is the content of the theorem. Likewise, rounding each coordinate of www separately to a fixed precision does not preserve the sign of w⋅bw \cdot bw⋅b when w⋅bw \cdot bw⋅b is tiny but nonzero.

Formalization scope

Vectors are functions on Fin n: the input www is rational (Fin n → ℚ) in the goal, in Theorem 3.3 and in the subset-sum corollary, as in the algorithm's input line; it is real in Theorem 3.1, Lemma 3.2 and Lemma 4.1, as on the page. Integral vectors are Fin n → ℤ, and AAA is a Matrix (Fin m) (Fin n) ℤ with every entry in {−1,0,1}\{-1, 0, 1\}{−1,0,1}, cast to R\mathbb{R}R; b∈Rmb \in \mathbb{R}^mb∈Rm is unrestricted. Decompositions are indexed by i∈{1,…,k}⊆Ni \in \{1, \dots, k\} \subseteq \mathbb{N}i∈{1,…,k}⊆N as in the paper. ∥b∥1\|b\|_1∥b∥1​ is always the explicit sum ∑j∣b(j)∣\sum_j |b(j)|∑j​∣b(j)∣, compared with N−1N - 1N−1 in Z\mathbb{Z}Z; ∥v∥∞\|v\|_\infty∥v∥∞​ of an integer vector is a natural number (supNorm). Sign equality uses SignType.sign and includes the zero case. Condition (iii) is stated multiplicatively together with vi≠0v_i \ne 0vi​=0, which the paper's quotient presupposes; without vi≠0v_i \ne 0vi​=0 Lemma 3.2 fails. An optimal dual basis is a maximal linearly independent set of row indices together with an optimal dual solution supported on it.

Two formalizations would make the goal trivial and are excluded: dropping the bound on ∥w~∥∞\|\tilde w\|_\infty∥w~∥∞​ (a multiple of www then works), and letting w~\tilde ww~ depend on AAA and bbb (the goal states ∃w~\exists \tilde w∃w~ before ∀A,b\forall A, b∀A,b). The 0,±10, \pm10,±1 assumption on AAA is part of every Section 4 statement.

A complete development needs a multidimensional pigeonhole argument, determinant and adjugate bounds for 0,±10, \pm 10,±1 matrices, and basic LP duality (strong duality, complementary slackness, basic optimal dual solutions); the last two are reusable across linear-programming missions. Proofs of individual milestones, reusable lemmas on LP duality, and alternative proofs of Dirichlet's theorem are all welcome.

Selected references

  • A. Frank and É. Tardos, An application of simultaneous diophantine approximation in combinatorial optimization, Combinatorica 7(1) (1987) 49–65. https://doi.org/10.1007/BF02579200
  • A. K. Lenstra, H. W. Lenstra Jr. and L. Lovász, Factoring polynomials with rational coefficients, Math. Ann. 261 (1982) 515–534. https://doi.org/10.1007/BF01457454
  • É. Tardos, A strongly polynomial algorithm to solve combinatorial linear programs, Operations Research 34(2) (1986) 250–256. https://doi.org/10.1287/opre.34.2.250
  • M. Grötschel, L. Lovász and A. Schrijver, The ellipsoid method and its consequences in combinatorial optimization, Combinatorica 1 (1981) 169–197. https://doi.org/10.1007/BF02579273
  • J. W. S. Cassels, An Introduction to the Theory of Numbers (title as printed in the paper's reference [2]), Springer, Berlin, 1971; cited in the paper as [2, Sect. 1.10] for Dirichlet's theorem.
11 thms5 active usersReviewed
🏆Completed
CombinatoricsTheoretical Computer Science·Captain: mikedeng1

Competitive Paging Algorithms IV: An Algorithm Competitive against Several Others Exists iff the Reciprocal Ratios Sum to at Most 1Research Paper

Motivation

Paging is the problem of managing a fast memory that holds kkk pages out of nnn: when a requested page is not in fast memory (a page fault), some resident page must be evicted, and the cost of an algorithm is its number of faults. Practitioners have many eviction rules. Least-recently-used (LRU) performs well on real workloads but can be kkk times worse than the optimal off-line schedule; the randomized marking algorithm of the same paper is 2Hk2H_k2Hk​-competitive and so has better worst-case guarantees. Fiat, Karp, Luby, McGeoch, Sleator and Young asked in 1991 whether one on-line algorithm can combine the advantages of several given ones, and answered the question exactly: the attainable combinations of ratios are characterized by one inequality (arXiv:cs/0205038, §6).

The question of combining on-line algorithms has since become a theme of its own: combining heuristics with worst-case-safe algorithms, and, more recently, combining machine-learned predictions with robust fallbacks, both ask for the same kind of guarantee against several reference algorithms at once.

Setting

A type (k,n)(k,n)(k,n) consists of kkk servers and a finite set MMM of nnn vertices with the uniform metric: two distinct vertices are at distance 111. This is paging: vertices are pages, the vertices covered by servers are the pages in fast memory, and a server move is a page fault.

A deterministic on-line algorithm AAA of type (k,n)(k,n)(k,n) has an initial configuration of its kkk servers and, after each request r∈Mr\in Mr∈M, moves servers so that some server covers rrr; its configuration after a request sequence depends only on that sequence. Its cost CA(σ)C_A(\sigma)CA​(σ) on a request sequence σ\sigmaσ is the total distance its servers travel, i.e. the number of server moves.

For algorithms AAA and BBB of the same type and a constant ccc, AAA is ccc-competitive against BBB if there is a constant aaa such that for every request sequence σ\sigmaσ

CA(σ)≤c⋅CB(σ)+a.C_A(\sigma)\le c\cdot C_B(\sigma)+a .CA​(σ)≤c⋅CB​(σ)+a.

A sequence c∗=(c(1),…,c(m))c^*=(c(1),\dots,c(m))c∗=(c(1),…,c(m)) of positive reals is realizable if for every type (k,n)(k,n)(k,n) and every mmm deterministic on-line algorithms B(1),…,B(m)B(1),\dots,B(m)B(1),…,B(m) of that type there is a deterministic on-line algorithm AAA of the same type that is c(i)c(i)c(i)-competitive against B(i)B(i)B(i) for every iii.

Formalization targets

Goal: Theorem 6

For m≥1m\ge1m≥1 and positive reals c(1),…,c(m)c(1),\dots,c(m)c(1),…,c(m),

c∗ is realizable  ⟺  ∑1≤i≤m1c(i)≤1.c^*\ \text{is realizable}\iff \sum_{1\le i\le m}\frac1{c(i)}\le 1 .c∗ is realizable⟺1≤i≤m∑​c(i)1​≤1.

Milestones

In the order of the paper's proof:

  1. Punishments are paid for. If AAA punishes BBB at a time step (an AAA-interval on a vertex vvv ends at that step and contains the end of a BBB-interval on vvv that began no later), then BBB has moved a server; the number of such steps is at most CB(σ)C_B(\sigma)CB​(σ).
  2. A fault leaves room to punish. If ∣SA∣=k|S_A|=k∣SA​∣=k, ∣SB∣≤k|S_B|\le k∣SB​∣≤k, x∈SBx\in S_Bx∈SB​ and x∉SAx\notin S_Ax∈/SA​, then some u∈SAu\in S_Au∈SA​ is not in SBS_BSB​.
  3. The greedy quota claim. If ∑i1/c(i)≤1\sum_i 1/c(i)\le 1∑i​1/c(i)≤1 and each unit of cost punishes the B(i)B(i)B(i) minimizing c(i)(PUN(i)+1)c(i)(\mathrm{PUN}(i)+1)c(i)(PUN(i)+1) (other algorithms may be punished incidentally), then after cost rrr every B(i)B(i)B(i) has been punished at least ⌊r/c(i)⌋\lfloor r/c(i)\rfloor⌊r/c(i)⌋ times.
  4. Shuttle algorithms. With 2m−12m-12m−1 servers on 2m2m2m vertices there are mmm algorithms, each keeping all vertices outside its own pair covered, no two of which move at the same step; in particular their total cost on any σ\sigmaσ is at most ∣σ∣|\sigma|∣σ∣.
  5. A forcing adversary. With 2m−12m-12m−1 servers on 2m2m2m vertices every algorithm can be forced to move at each of NNN steps, so CA(τ(N))≥NC_A(\tau(N))\ge NCA​(τ(N))≥N.

Significance

The result. Theorem 6 is an exact characterization, not a bound: the region of simultaneously attainable ratios against arbitrary deterministic paging algorithms is {c:∑1/c(i)≤1}\{c:\sum 1/c(i)\le 1\}{c:∑1/c(i)≤1}. For example, any two paging algorithms can be combined into one that is 222-competitive against each, and no better symmetric pair is possible in general. Combined with Theorem 7 of the same paper (not part of this mission), the same region is attainable against randomized algorithms, which is how LRU's practical behaviour and the marking algorithm's 2Hk2H_k2Hk​ worst-case guarantee can be obtained within constant factors by one algorithm.

Formalizing it. The theorem has been proved since 1991; no machine-checked proof is known. A formal proof produces a reusable notion of competitiveness of one on-line algorithm against another, built on the published KServer_model definitions, and a formal account of the scheduling fact at the core of the sufficiency proof.

Difficulty

Sufficiency looks like an averaging argument, but the combined algorithm cannot simulate the B(i)B(i)B(i) and follow one of them: switching between their configurations costs up to kkk per switch, which no additive constant absorbs. The accounting has to charge each of AAA's faults to a specific move of a specific B(i)B(i)B(i), and the charge must be injective; the paper's claim that CB(σ)C_B(\sigma)CB​(σ) is at least the number of punishments is where this happens, and it depends on how server intervals are matched. The allocation of faults to algorithms is then a deadline-scheduling problem whose feasibility is exactly ∑1/c(i)≤1\sum 1/c(i)\le 1∑1/c(i)≤1, and the floor functions make the counting delicate at the boundary. The paper's own definition of punishment only counts intervals that start with a move, so the first kkk faults of AAA (servers on their initial vertices) need separate treatment; they are absorbed by the additive constant.

Necessity needs the right family of hard instances: the mmm algorithms must never move at the same step, which pins the type to (2m−1,2m)(2m-1,2m)(2m−1,2m).

Formalization scope

The Lean development works in the namespace CompetitivePaging.Combining and imports the published KServer_model definitions: KServer.OnlineAlgorithm k M (a configuration map from request prefixes to Fin k → M with a serving condition) and OnlineAlgorithm.cost. Committed conventions:

  • a type (k,n)(k,n)(k,n) is any k : ℕ and any finite M : Type with a metric in which distinct points are at distance 111; realizability quantifies over all of them, never over one fixed type;
  • servers are labelled; each algorithm has its own initial configuration, and the additive constant aaa is chosen before the request sequence;
  • c(i)>0c(i)>0c(i)>0 and m≥1m\ge1m≥1 are hypotheses of the goal, as in the paper; without positivity, 1/0=01/0=01/0=0 in Lean would make a zero ratio free;
  • time ttt is the step processing the ttt-th request; the paper's PUN\mathrm{PUN}PUN counts time steps.

Trivializing encodings are ruled out: realizability is not stated for a single fixed type, the metric is not the metric of Fin n, and the competitive constant is not allowed to depend on the request sequence.

A complete proof needs the construction of the punishing algorithm as a KServer.OnlineAlgorithm (a lazy, injective algorithm whose moves depend on the prefix and on the B(i)B(i)B(i)'s configurations), the injective charging argument, the scheduling lemma, and the explicit shuttle algorithms. The scheduling lemma and the charging lemma are independent of paging and reusable. Proofs of any milestone are welcome, as are alternative statements of the sufficiency construction.

Selected references

  • A. Fiat, R. M. Karp, M. Luby, L. A. McGeoch, D. D. Sleator, N. E. Young, Competitive Paging Algorithms, J. Algorithms 12(4):685–699, 1991. doi:10.1016/0196-6774(91)90041-V; preprint arXiv:cs/0205038.
  • D. D. Sleator, R. E. Tarjan, Amortized efficiency of list update and paging rules, Comm. ACM 28(2):202–208, 1985. doi:10.1145/2786.2793
  • M. S. Manasse, L. A. McGeoch, D. D. Sleator, Competitive algorithms for server problems, J. Algorithms 11(2):208–230, 1990. doi:10.1016/0196-6774(90)90003-W
9 thms5 active usersReviewed
🏆Completed
CombinatoricsGraph TheoryLinear Optimization+1·Captain: mikedeng1

Theoretical Improvements in Algorithmic Efficiency for Network Flow Problems 3: Capacity Scaling for the Hitchcock ProblemResearch Paper

Motivation

The Hitchcock transportation problem asks how to ship a commodity from mmm supply points to nnn demand points at minimum total cost. It was posed by Hitchcock in 1941 and is one of the founding problems of linear programming and network optimization; it is solved routinely in logistics, and its structure (a bipartite network with supplies, demands and per-unit costs) recurs in assignment, optimal transport and matching.

The classical algorithms for it, the Ford–Fulkerson primal–dual method among them, augment flow one path at a time. With integral data their number of augmentations is bounded only by the total supply ∑iai\sum_i a_i∑i​ai​, which is exponential in the number of binary digits used to write the data. Edmonds and Karp, Theoretical Improvements in Algorithmic Efficiency for Network Flow Problems (J. ACM 19(2), 1972, doi:10.1145/321694.321699), introduced capacity scaling: solve a coarse version of the problem first, then refine one binary digit at a time. Their Theorem 9 (p. 260) bounds the total number of augmentations by a quantity proportional to max⁡(m,n)\max(m,n)max(m,n) times the number of bits of the data, which made the transportation problem, and through standard reductions the minimum-cost flow problem, one of the first network problems with a polynomial-time ("good") algorithm in the sense of Edmonds.

Timeline. Hitchcock (1941) posed the problem; Ford and Fulkerson (1956–1962) gave the primal–dual labeling method and the optimality conditions by node potentials; Edmonds and Karp (1972) gave the scaling method and the bound formalized here. Strongly polynomial algorithms, independent of the size of the numbers, came later (Tardos 1985; Orlin 1988).

Setting

The network of Figure 1 (p. 259) has a source sss, a sink ttt, supply nodes s1,…,sms_1,\dots,s_ms1​,…,sm​ and demand nodes t1,…,tnt_1,\dots,t_nt1​,…,tn​, with m,n≥1m,n\ge 1m,n≥1. Its arcs are (s,si)(s,s_i)(s,si​) with capacity aia_iai​ and cost 000; (si,tj)(s_i,t_j)(si​,tj​) with capacity +∞+\infty+∞ and cost dij≥0d_{ij}\ge 0dij​≥0; (tj,t)(t_j,t)(tj​,t) with capacity bjb_jbj​ and cost 000; and the return arc (t,s)(t,s)(t,s) with capacity +∞+\infty+∞ and cost 000. The supplies aia_iai​ and demands bjb_jbj​ are positive integers with ∑iai=∑jbj=:B\sum_i a_i=\sum_j b_j=:B∑i​ai​=∑j​bj​=:B.

A flow assigns a nonnegative number to every arc, at most the capacity, with inflow equal to outflow at every node. Write f0i=f(s,si)f_{0i}=f(s,s_i)f0i​=f(s,si​), fij=f(si,tj)f_{ij}=f(s_i,t_j)fij​=f(si​,tj​), fj0=f(tj,t)f_{j0}=f(t_j,t)fj0​=f(tj​,t); the value of fff is f(t,s)f(t,s)f(t,s), and a maximum flow is one of largest value. Its cost is ∑i,jdijfij\sum_{i,j} d_{ij} f_{ij}∑i,j​dij​fij​; a flow is extreme if no flow of the same value is cheaper. A flow is pseudo-extreme if there are real ui,vju_i, v_jui​,vj​ with ui−vj+dij≥0u_i-v_j+d_{ij}\ge 0ui​−vj​+dij​≥0 for all i,ji,ji,j and fij=0f_{ij}=0fij​=0 whenever ui−vj+dij>0u_i-v_j+d_{ij}>0ui​−vj​+dij​>0.

An augmenting path relative to fff is a sequence of distinct nodes from sss to ttt in which each step either follows an arc with spare capacity or traverses backwards an arc carrying positive flow; augmenting pushes the minimum spare amount ε\varepsilonε along it and raises f(t,s)f(t,s)f(t,s) by ε\varepsilonε.

For p≥0p\ge 0p≥0, Problem ppp has the same network and costs, with capacities ⌊ai/2p⌋\lfloor a_i/2^p\rfloor⌊ai​/2p⌋ and ⌊bj/2p⌋\lfloor b_j/2^p\rfloor⌊bj​/2p⌋. Choose lll with every ai,bj<2la_i,b_j<2^lai​,bj​<2l. The scaling method solves Problems l−1,l−2,…,0l-1,l-2,\dots,0l−1,l−2,…,0 in turn. Each phase performs augmentations keeping every flow pseudo-extreme, until no augmenting path is left. Problem l−1l-1l−1 starts from the zero flow, and Problem p−1p-1p−1 starts from twice the final flow of Problem ppp.

Formalization targets

Goal: Theorem 9

For every run of the scaling method, the total number ∑p<lKp\sum_{p<l} K_p∑p<l​Kp​ of flow augmentations satisfies

∑p=0l−1Kp  ≤  max⁡(m,n)(2+⌊log⁡2∑i=1maimax⁡(m,n)⌋).\sum_{p=0}^{l-1} K_p \;\le\; \max(m,n)\left(2+\left\lfloor \log_2\frac{\sum_{i=1}^m a_i}{\max(m,n)}\right\rfloor\right).p=0∑l−1​Kp​≤max(m,n)(2+⌊log2​max(m,n)∑i=1m​ai​​⌋).

The bound holds for every lll admissible for the data and every choice of costs, and is stated with the paper's constant exactly.

Milestones

  1. §1.1: augmentation preserves feasibility and raises the value by ε>0\varepsilon>0ε>0; a flow is maximum iff no augmenting path exists.
  2. Theorem 8: a maximum flow is extreme iff there are potentials u0,…,umu_0,\dots,u_mu0​,…,um​, v0,…,vnv_0,\dots,v_nv0​,…,vn​ with (5a)–(5f).
  3. §2.2: a pseudo-extreme maximum flow is extreme.
  4. Lemma 3: if fff is pseudo-extreme in Problem ppp, then 2f2f2f is pseudo-extreme in Problem p−1p-1p−1.
  5. The maximum-flow value of Problem ppp is fp∗=min⁡(∑i⌊ai/2p⌋,∑j⌊bj/2p⌋)f_p^*=\min\big(\sum_i\lfloor a_i/2^p\rfloor,\sum_j\lfloor b_j/2^p\rfloor\big)fp∗​=min(∑i​⌊ai​/2p⌋,∑j​⌊bj​/2p⌋).
  6. Eq. (6): ∑pKp≤f0∗−∑p=1l−1fp∗\sum_p K_p\le f_0^*-\sum_{p=1}^{l-1} f_p^*∑p​Kp​≤f0∗​−∑p=1l−1​fp∗​.
  7. fp∗≥max⁡(0, B/2p−max⁡(m,n))f_p^*\ge\max\big(0,\,B/2^p-\max(m,n)\big)fp∗​≥max(0,B/2p−max(m,n)).

Significance

The result. Theorem 9 shows that scaling reduces the number of augmentations from order BBB to order max⁡(m,n)log⁡2(B/max⁡(m,n))\max(m,n)\log_2(B/\max(m,n))max(m,n)log2​(B/max(m,n)), which is roughly the length of the binary encoding of the data. Combined with the O(mn)O(mn)O(mn) cost of one augmentation it gives a polynomial-time algorithm for the transportation problem; through the reduction of minimum-cost flow to transportation (p. 261) it gives one for minimum-cost flow. Capacity and cost scaling became standard techniques in network optimization and in combinatorial optimization generally. Theorem 8 and the pseudo-extreme criterion are the optimality certificates for transportation, a special case of linear-programming complementary slackness.

Formalizing it. The theorem has been proved since 1972. This mission produces a machine-checked version of the bound, of the exact counting argument (eq. (6)) and of the arithmetic estimate that turns it into the stated constant, together with the potential-based optimality conditions for the transportation network. To our knowledge no machine-checked bound on the number of augmentations of a flow algorithm exists on the platform.

Difficulty

The obvious argument bounds the number of augmentations by the increase in flow value, since each augmentation raises the value by a positive integer. Applied to Problem 0 directly this gives only BBB. The scaling bound needs three things that the naive count does not supply. First, doubling the final flow of Problem ppp must give a feasible, still pseudo-extreme, starting flow for Problem p−1p-1p−1 (Lemma 3). Second, the gap between that start and the optimum of Problem p−1p-1p−1 must be small, which requires the exact maximum-flow value fp∗f_p^*fp∗​ of each scaled problem. Third, the telescoping sum of the gaps must be estimated against log⁡2(B/max⁡(m,n))\log_2(B/\max(m,n))log2​(B/max(m,n)) with the floors handled exactly. Integrality of every intermediate flow is not assumed; it has to be carried along the run from the integral capacities and the zero start.

Formalization scope

Everything lives in the namespace EdmondsKarp.Scaling. Nodes form an inductive type (s, t, src i, dst j). A flow is a structure with components f0, fx, fz, ret for the four arc families, real valued; the infinite capacities are encoded by the absence of an upper bound. IsMaxFlow is a predicate comparing values with every flow, not a supremum. Problem ppp uses natural-number division for ⌊ai/2p⌋\lfloor a_i/2^p\rfloor⌊ai​/2p⌋. Augmenting paths are lists of distinct nodes from s to t whose consecutive pairs have positive residual amount (resCap, valued in WithTop ℝ); they never use the return arc.

A run of the scaling method (IsScalingRun) is a family of phases F p 0,…,F p (K p)F\,p\,0,\dots,F\,p\,(K\,p)Fp0,…,Fp(Kp) for p<lp<lp<l. It starts from 000 in Problem l−1l-1l−1, restarts from 2F p (K p)2F\,p\,(K\,p)2Fp(Kp) in Problem p−1p-1p−1, and advances by one augmentation per step. Every flow is pseudo-extreme, and each phase ends with no augmenting path left. The paper's path-selection rule (minimum weight for the modified reduced costs Δˉ\bar\DeltaΔˉ) is abstracted to this invariant, which the paper states for it, so every run of the paper's method is covered.

The goal compares the count, cast to Z\mathbb{Z}Z, with max⁡(m,n) (2+⌊log⁡2(B/max⁡(m,n))⌋)\max(m,n)\,(2+\lfloor\log_2(B/\max(m,n))\rfloor)max(m,n)(2+⌊log2​(B/max(m,n))⌋), using Real.logb 2 and Int.floor. The constant is the printed one; neither O(⋅)O(\cdot)O(⋅) nor a weaker constant is acceptable. Positivity of all ai,bja_i,b_jai​,bj​ is a hypothesis, because without it the printed bound is false (for m=5m=5m=5, n=1n=1n=1, a=(1,0,0,0,0)a=(1,0,0,0,0)a=(1,0,0,0,0), b=(1)b=(1)b=(1) one augmentation is needed and the bound is negative). A formalization in which runs could be empty or never reach a maximum flow would trivialize the goal; the run predicate forbids this, and it is satisfiable (for instance with m=n=1m=n=1m=n=1, a=b=(1)a=b=(1)a=b=(1), l=1l=1l=1 and one augmentation).

A complete development needs max-flow/min-cut for the bipartite network, integrality of flows along a run, the exact value of fp∗f_p^*fp∗​, the telescoping identity and floor/logarithm estimates. LP duality or complementary slackness is needed only for Theorem 8. The augmenting-path and max-flow lemmas are reusable for other bipartite flow problems. Contributions welcome: proofs of any milestone, and a formalization of the paper's exact Δˉ\bar\DeltaΔˉ path rule showing that it satisfies the invariant.

Selected references

  • J. Edmonds, R. M. Karp, Theoretical Improvements in Algorithmic Efficiency for Network Flow Problems, Journal of the ACM 19(2):248–264, 1972. https://doi.org/10.1145/321694.321699
  • F. L. Hitchcock, The Distribution of a Product from Several Sources to Numerous Localities, Journal of Mathematics and Physics 20:224–230, 1941. https://doi.org/10.1002/sapm1941201224
  • L. R. Ford, D. R. Fulkerson, Flows in Networks, Princeton University Press, 1962. https://doi.org/10.1515/9781400875184
  • É. Tardos, A strongly polynomial minimum cost circulation algorithm, Combinatorica 5:247–255, 1985. https://doi.org/10.1007/BF02579369
  • J. B. Orlin, A faster strongly polynomial minimum cost flow algorithm, Proc. STOC 1988, 377–387. https://doi.org/10.1145/62212.62249
12 thms5 active usersReviewed
🏆Completed
OptimizationProbability·Captain: mikedeng1

On Properties of Stochastic Inventory Systems I: Using the EOQ Order Quantity in the Stochastic (Q, r) Model Raises Costs by at Most 1/8Research Paper

Motivation

The economic order quantity (EOQ) is the most widely used formula in inventory management. It assumes that demand is a deterministic constant stream. Real demand is random, and the model that accounts for this, the continuous-review (Q,r)(Q, r)(Q,r) policy with stochastic demand, has no closed-form optimum: for decades its optimal parameters were computed numerically, one instance at a time, which gave little insight into how the stochastic system behaves. Practitioners nevertheless kept using the EOQ quantity in stochastic systems and observed that the cost penalty was small (Wagner, O'Hagan and Lundh 1965; Naddor 1975; Archibald and Silver 1978), without an analytical explanation.

Zheng (Management Science 38(1), 1992) supplied that explanation. By minimising the average cost first over the reorder point and then over the order quantity, he obtained two simple optimality equations and compared the stochastic model with the EOQ model under the same cost structure. One of the results is the goal of this mission: for every leadtime-demand distribution, using the EOQ order quantity in the stochastic system raises the average cost by at most one eighth. The paper extends Federgruen and Zheng (1988), who treated the discrete-demand version of the cost function.

Setting

A single item faces demand at rate λ>0\lambda > 0λ>0. Orders are delivered after a fixed leadtime L>0L > 0L>0, and stockouts are backordered. Each order costs a fixed K>0K > 0K>0. Inventory is held at cost rate h>0h > 0h>0 per unit and backorders are penalised at cost rate p>0p > 0p>0 per unit. Let D≥0D \ge 0D≥0 be the demand during a leadtime, with mean E(D)=λLE(D) = \lambda LE(D)=λL. The newsvendor cost

G(y)=E[h(y−D)++p(D−y)+]G(y) = E\big[h(y - D)^+ + p(D - y)^+\big]G(y)=E[h(y−D)++p(D−y)+]

is the rate at which expected inventory costs accrue at time t+Lt + Lt+L when the inventory position at time ttt is yyy. It is convex, and it is assumed, as in the paper, to attain its minimum at a unique point y0y^0y0.

A (Q,r)(Q, r)(Q,r) policy orders QQQ units whenever the inventory position falls to the reorder point rrr. Its long-run average cost is

c(Q,r)=λK+∫rr+QG(y) dyQ.(1)c(Q, r) = \frac{\lambda K + \int_r^{r+Q} G(y)\,dy}{Q}. \qquad (1)c(Q,r)=QλK+∫rr+Q​G(y)dy​.(1)

For a fixed Q>0Q > 0Q>0 let r(Q)r(Q)r(Q) be an optimal reorder point, and set

H(Q)=G(r(Q)) (Q>0),H(0)=G(y0),C(Q)=c(Q,r(Q)),A(Q)=QH(Q)−∫0QH(y) dy.H(Q) = G(r(Q)) \ (Q > 0),\quad H(0) = G(y^0),\quad C(Q) = c(Q, r(Q)),\quad A(Q) = Q H(Q) - \int_0^Q H(y)\,dy .H(Q)=G(r(Q)) (Q>0),H(0)=G(y0),C(Q)=c(Q,r(Q)),A(Q)=QH(Q)−∫0Q​H(y)dy.

C(Q)C(Q)C(Q) is the average cost when the reorder point is chosen optimally for QQQ; an order quantity Q∗>0Q^* > 0Q∗>0 minimising CCC over Q>0Q > 0Q>0 is the optimal order quantity, and C∗=C(Q∗)C^* = C(Q^*)C∗=C(Q∗) is the optimal cost.

The EOQ model is the special case in which the leadtime demand is the constant λL\lambda LλL. Its cost rate is Gd(y)=h(y−λL)++p(λL−y)+G_d(y) = h(y - \lambda L)^+ + p(\lambda L - y)^+Gd​(y)=h(y−λL)++p(λL−y)+, and its optimal order quantity is

Qd∗=2λK(h+p)hp.Q^*_d = \sqrt{\frac{2\lambda K(h+p)}{hp}} .Qd∗​=hp2λK(h+p)​​.

The subscript ddd marks every object of the EOQ model (rdr_drd​, HdH_dHd​, AdA_dAd​).

Formalization targets

Goal: Theorem 5

R=C(Qd∗)−C∗C∗  ≤  18−12(12−Qd∗Q∗)2  ≤  18.R = \frac{C(Q^*_d) - C^*}{C^*} \;\le\; \frac18 - \frac12\left(\frac12 - \frac{Q^*_d}{Q^*}\right)^2 \;\le\; \frac18 .R=C∗C(Qd∗​)−C∗​≤81​−21​(21​−Q∗Qd∗​​)2≤81​.

C(Qd∗)C(Q^*_d)C(Qd∗​) is the stochastic cost at the EOQ quantity with the reorder point re-optimised for it. The bound holds for every leadtime-demand distribution and every value of the parameters.

Milestones

In the order the goal's proof uses them:

  • Lemma 1 (joint convexity of ccc), Lemma 2 (r=r(Q)  ⟺  G(r)=G(r+Q)r = r(Q) \iff G(r) = G(r+Q)r=r(Q)⟺G(r)=G(r+Q)), Lemma 3 (properties of r(Q)r(Q)r(Q)), Corollary 1 (G(r(Q))≤max⁡(G(r),G(r+Q))G(r(Q)) \le \max(G(r), G(r+Q))G(r(Q))≤max(G(r),G(r+Q)));
  • Eq. (7) (C(Q)=(λK+∫0QH)/QC(Q) = (\lambda K + \int_0^Q H)/QC(Q)=(λK+∫0Q​H)/Q), Lemma 4 (HHH increasing, convex, asymptotic slope hp/(h+p)hp/(h+p)hp/(h+p)), Lemma 5 (CCC convex), Eq. (8) (H(Q∗)=C(Q∗)H(Q^*) = C(Q^*)H(Q∗)=C(Q∗)), Theorem 1 ((Q,r)(Q, r)(Q,r) optimal iff c(Q,r)=G(r)=G(r+Q)c(Q,r) = G(r) = G(r+Q)c(Q,r)=G(r)=G(r+Q)), Lemma 6 (AAA increasing convex, Q=Q∗  ⟺  A(Q)=λKQ = Q^* \iff A(Q) = \lambda KQ=Q∗⟺A(Q)=λK, comparative statics in KKK);
  • Eqs. (18), (20) (the EOQ model: Hd(Q)=hpQ/(h+p)H_d(Q) = hpQ/(h+p)Hd​(Q)=hpQ/(h+p) and the formula for Qd∗Q^*_dQd∗​), Eq. (22) (Gd≤GG_d \le GGd​≤G), Lemma 7 (H0≤Hd≤HH_0 \le H_d \le HH0​≤Hd​≤H, A≤AdA \le A_dA≤Ad​), and the first inequality of Theorem 2, Qd∗≤Q∗Q^*_d \le Q^*Qd∗​≤Q∗.

Significance

The theorem is a distribution-free worst-case guarantee for the most common heuristic in inventory practice. It says that the EOQ formula, which needs only the mean demand rate and three cost parameters, loses at most 12.5%12.5\%12.5% against the true optimum, and that the loss is smaller when Qd∗/Q∗Q^*_d/Q^*Qd∗​/Q∗ is close to 12\frac1221​ or to 111. Combined with the paper's Theorem 2 (the gap Q∗−Qd∗Q^* - Q^*_dQ∗−Qd∗​ is bounded as KKK grows), it implies that the relative loss vanishes as the fixed ordering cost grows. The milestones along the way (the optimality conditions of Theorem 1, the monotonicity of the optimal reorder point, the comparison of HHH with its EOQ counterpart) are the standard structural facts about continuous (Q,r)(Q, r)(Q,r) systems and are reused throughout the inventory literature.

The result is proved in the paper. No machine-checked version of it, or of the (Q,r)(Q, r)(Q,r) optimality conditions for the continuous cost (1), is known to exist. The platform has related but different formalizations: the discrete cost function with integer QQQ (InventoryControl_rqDiscrete), the (R,Q)(R, Q)(R,Q) cost under normal demand (InventoryControl_rq), and the EOQ without backorders (InventoryControl_eoq). A complete development here provides the continuous (Q,r)(Q, r)(Q,r) machinery for an arbitrary leadtime-demand distribution.

Difficulty

The paper's proofs differentiate r(Q)r(Q)r(Q) and H(Q)H(Q)H(Q) twice (Eqs. (4), (5)), which presupposes a leadtime demand with a smooth density. The formalization does not assume one, so that discrete demand, such as the Poisson demand of the paper's own numerical study, is covered. Every step that the paper takes through derivatives, in particular the convexity of HHH and the slope bound H′≤hp/(h+p)H' \le hp/(h+p)H′≤hp/(h+p), has to be established by other means: through one-sided derivatives or chord arguments for the convex function GGG, whose kinks are where the derivative-based argument breaks. The definition of r(Q)r(Q)r(Q) as a minimiser also means its existence and uniqueness must be proved before any property of HHH, CCC or AAA can be used.

Formalization scope

  • The model is a probability measure μ\muμ on R\mathbb{R}R (the law of DDD), concentrated on [0,∞)[0,\infty)[0,∞), integrable, with mean λL\lambda LλL; GGG is the integral of hmax⁡(y−x,0)+pmax⁡(x−y,0)h\max(y-x,0) + p\max(x-y,0)hmax(y−x,0)+pmax(x−y,0) against μ\muμ. The standing assumptions are bundled in IsQRModel: λ,L,h,p>0\lambda, L, h, p > 0λ,L,h,p>0, the conditions on μ\muμ, and the unique minimiser of GGG (paper, p. 90). K>0K > 0K>0 is a separate hypothesis. No density is assumed.
  • ccc, r(Q)r(Q)r(Q), y0y^0y0, HHH, H0H_0H0​, CCC, AAA and optimality of an order quantity are defined for an arbitrary cost rate GGG and applied both to the newsvendor cost and to GdG_dGd​; the EOQ objects rdr_drd​, HdH_dHd​, AdA_dAd​ are these definitions at GdG_dGd​, and Eqs. (18), (20) are theorems.
  • r(Q)r(Q)r(Q) is a chosen minimiser of c(Q,⋅)c(Q,\cdot)c(Q,⋅) (never defined by G(r)=G(r+Q)G(r) = G(r+Q)G(r)=G(r+Q), which is Lemma 2). H(0):=G(y0)H(0) := G(y^0)H(0):=G(y0); right-continuity of HHH at 000 is part of the Lemma 4 milestone. Order quantities range over (0,∞)(0,\infty)(0,∞) for ccc and CCC and over [0,∞)[0,\infty)[0,∞) for HHH, H0H_0H0​, AAA.
  • Q∗Q^*Q∗ in the goal is any Q>0Q > 0Q>0 with C(Q)≤C(Q′)C(Q) \le C(Q')C(Q)≤C(Q′) for all Q′>0Q' > 0Q′>0; Lemma 6 asserts that exactly one exists, so the goal is not vacuous. Qd∗Q^*_dQd∗​ is the explicit square-root formula.
  • Readings of informal words: "increasing" in Lemmas 4 and 6 means strictly increasing on [0,∞)[0,\infty)[0,∞); "Q∗Q^*Q∗ increasing, r∗r^*r∗ decreasing in KKK" means strictly; "asymptotic slope hp/(h+p)hp/(h+p)hp/(h+p)" means that every chord of HHH on [0,∞)[0,\infty)[0,∞) has slope at most hp/(h+p)hp/(h+p)hp/(h+p) and H(Q)/Q→hp/(h+p)H(Q)/Q \to hp/(h+p)H(Q)/Q→hp/(h+p); Lemma 3 part 3 is stated as strict monotonicity of r(Q)r(Q)r(Q) and r(Q)+Qr(Q) + Qr(Q)+Q, and its derivative clause −1<r′(Q)<0-1 < r'(Q) < 0−1<r′(Q)<0 is omitted because rrr need not be differentiable without a density; "the optimal order quantity" means existence and uniqueness; Lemma 7 is stated for Q≥0Q \ge 0Q≥0.
  • A trivializing formalization is ruled out: C(Qd∗)C(Q^*_d)C(Qd∗​) is the stochastic cost with the reorder point re-optimised in the stochastic model (not the EOQ reorder point rd∗r^*_drd∗​), and no positivity of C∗C^*C∗ is assumed (it follows from the model).
  • Out of scope: the discrete cost (2), the §4 numerical study, and the unnumbered remarks after Theorem 5.
  • Needed infrastructure: interval integrals of convex functions, partial minimisation of jointly convex functions, Jensen's inequality for μ\muμ, and one-sided derivatives of convex functions. The generic (Q,r)(Q, r)(Q,r) machinery (optimality conditions for arbitrary convex GGG) is reusable for other continuous-review models. Proofs of any milestone, and alternative proofs that avoid the paper's differentiability assumptions, are welcome.

Selected references

  • Yu-Sheng Zheng, On Properties of Stochastic Inventory Systems, Management Science 38(1):87–103, 1992. https://doi.org/10.1287/mnsc.38.1.87
  • Awi Federgruen and Yu-Sheng Zheng, An Efficient Algorithm for Computing an Optimal (r, Q) Policy in Continuous Review Stochastic Inventory Systems, Operations Research 40(4):808–813, 1992. https://doi.org/10.1287/opre.40.4.808
  • Paul H. Zipkin, Inventory Service-Level Measures: Convexity and Approximation, Management Science 32(8):975–981, 1986. https://doi.org/10.1287/mnsc.32.8.975
  • George Hadley and Thomson M. Whitin, Analysis of Inventory Systems, Prentice-Hall, 1963.
  • Daniel P. Heyman and Matthew J. Sobel, Stochastic Models in Operations Research, Vol. II, McGraw-Hill, 1984.
18 thms5 active usersReviewed
🏆Completed
AnalysisOptimization·Captain: mikedeng1

Robust Mean-Covariance Solutions for Stochastic Optimization II: An Inverse S-Shaped Derivative with Finite Limits Gives the Two-Point Support PropertyResearch Paper

Motivation

In stochastic optimization the law of a random return rrr is rarely known exactly, while its mean and variance can be estimated. A robust mean-covariance decision maker therefore evaluates a utility uuu by its worst case

U=inf⁡{ Eν[u(r)]:ν a law on R with mean μ and variance σ2 }.U=\inf\{\,E_\nu[u(r)] : \nu \text{ a law on } \mathbb R \text{ with mean } \mu \text{ and variance } \sigma^2\,\}.U=inf{Eν​[u(r)]:ν a law on R with mean μ and variance σ2}.

Popescu (Operations Research 55(1), 2007) shows that for large classes of utilities this infinite-dimensional problem collapses to a one-dimensional one. The paper projects multivariate problems to a single dimension (the subject of the first mission of this series) and then asks for which uuu the univariate worst case sits on laws with two support points. This mission formalizes the answer the paper gives for utilities whose marginal utility u′u'u′ is decreasing and changes curvature once, with finite limits: log-logistic utilities C+log⁡11+e−axC+\log\frac{1}{1+e^{-ax}}C+log1+e−ax1​ used in statistics and classification, and catenary-type utilities C−bcosh⁡(ax)C-b\cosh(ax)C−bcosh(ax), each plus a concave quadratic.

The result has a bounded-support precursor: Birge and Dulá (Annals of Operations Research 30, 1991, Theorem 5.1, as cited on p. 102 of Popescu 2007) proved an analogous two-point statement for functions on a bounded interval. Popescu's Proposition 5 is the unbounded version on the whole real line.

Setting

Let u:R→Ru:\mathbb R\to\mathbb Ru:R→R.

  • The family Q\mathcal QQ collects the coefficient triples of quadratics lying below uuu: Q={(A,B,C)∣q(y)=Ay2+By+C≤u(y) ∀y∈R}\mathcal Q=\{(A,B,C) \mid q(y)=Ay^2+By+C\le u(y)\ \forall y\in\mathbb R\}Q={(A,B,C)∣q(y)=Ay2+By+C≤u(y) ∀y∈R}.
  • A two-point law with support {a,b}\{a,b\}{a,b}, a<ba<ba<b, puts mass p∈(0,1)p\in(0,1)p∈(0,1) on aaa and 1−p1-p1−p on bbb. It has mean μ\muμ and variance σ2\sigma^2σ2 when pa+(1−p)b=μpa+(1-p)b=\mupa+(1−p)b=μ and p(a−μ)2+(1−p)(b−μ)2=σ2p(a-\mu)^2+(1-p)(b-\mu)^2=\sigma^2p(a−μ)2+(1−p)(b−μ)2=σ2.
  • Two-point support property (Definition 1). uuu has it with respect to (μ,σ2)(\mu,\sigma^2)(μ,σ2) if some quadratic qqq with coefficients in Q\mathcal QQ meets uuu at two points a<ba<ba<b, i.e. q(a)=u(a)q(a)=u(a)q(a)=u(a), q(b)=u(b)q(b)=u(b)q(b)=u(b), and a two-point law with support {a,b}\{a,b\}{a,b}, mean μ\muμ and variance σ2\sigma^2σ2 exists. uuu has the two-point support property if this holds for every μ∈R\mu\in\mathbb Rμ∈R and every σ>0\sigma>0σ>0.
  • Shapes (Definition 2). f:R→Rf:\mathbb R\to\mathbb Rf:R→R is convex-concave if for some x0x_0x0​ it is convex on (−∞,x0)(-\infty,x_0)(−∞,x0​) and concave on (x0,∞)(x_0,\infty)(x0​,∞); concave-convex if −f-f−f is convex-concave; S-shaped if increasing and convex-concave; inverse S-shaped if −f-f−f is S-shaped. So an inverse S-shaped fff is decreasing, concave on (−∞,x0)(-\infty,x_0)(−∞,x0​) and convex on (x0,∞)(x_0,\infty)(x0​,∞).
  • Lemma 1's quadratic. For a<ba<ba<b and slopes qa,qbq_a,q_bqa​,qb​, set
A=qb−qa2(b−a),B=bqa−aqbb−a,C=bu(a)−au(b)b−a−ab qa−qb2(b−a),A=\frac{q_b-q_a}{2(b-a)},\quad B=\frac{bq_a-aq_b}{b-a},\quad C=\frac{bu(a)-au(b)}{b-a}-ab\,\frac{q_a-q_b}{2(b-a)},A=2(b−a)qb​−qa​​,B=b−abqa​−aqb​​,C=b−abu(a)−au(b)​−ab2(b−a)qa​−qb​​,

and q(y)=Ay2+By+Cq(y)=Ay^2+By+Cq(y)=Ay2+By+C (lemma1Quad).

  • The function ggg. For μ∈R\mu\in\mathbb Rμ∈R, σ>0\sigma>0σ>0 and y<μy<\muy<μ let z=μ+σ2/(μ−y)z=\mu+\sigma^2/(\mu-y)z=μ+σ2/(μ−y) (partnerPoint) and g(y)=u(z)−u(y)z−y−u′(y)+u′(z)2g(y)=\frac{u(z)-u(y)}{z-y}-\frac{u'(y)+u'(z)}{2}g(y)=z−yu(z)−u(y)​−2u′(y)+u′(z)​ (prop5Gap).

Formalization targets

Goal: Proposition 5 (p. 102)

If uuu is differentiable, u′u'u′ is inverse S-shaped, and the limits lim⁡y→−∞u′(y)\lim_{y\to-\infty}u'(y)limy→−∞​u′(y) and lim⁡y→+∞u′(y)\lim_{y\to+\infty}u'(y)limy→+∞​u′(y) exist and are finite, then

u satisfies the two-point support property.u \text{ satisfies the two-point support property.}u satisfies the two-point support property.

Milestones

  1. Proof of Lemma 1, first sentence. If a<ba<ba<b and u(b)−u(a)b−a=qa+qb2\frac{u(b)-u(a)}{b-a}=\frac{q_a+q_b}{2}b−au(b)−u(a)​=2qa​+qb​​, then q(a)=u(a)q(a)=u(a)q(a)=u(a) and q(b)=u(b)q(b)=u(b)q(b)=u(b).
  2. Tangency (p. 110). q′(a)=qaq'(a)=q_aq′(a)=qa​ and q′(b)=qbq'(b)=q_bq′(b)=qb​.
  3. Lemma 1. uuu has two-point support if and only if for all μ\muμ and σ>0\sigma>0σ>0 there are a<ba<ba<b and qa,qbq_a,q_bqa​,qb​ with
(b−μ)(μ−a)=σ2,u(b)−u(a)b−a=qa+qb2,q≤u on R.(b-\mu)(\mu-a)=\sigma^2,\qquad \frac{u(b)-u(a)}{b-a}=\frac{q_a+q_b}{2},\qquad q\le u \text{ on } \mathbb R.(b−μ)(μ−a)=σ2,b−au(b)−u(a)​=2qa​+qb​​,q≤u on R.
  1. Limits of ggg (p. 110). Under the goal's hypotheses, with ℓ±=lim⁡y→±∞u′(y)\ell_\pm=\lim_{y\to\pm\infty}u'(y)ℓ±​=limy→±∞​u′(y),
lim⁡y→−∞g(y)=ℓ−−u′(μ)2>0,lim⁡y→μ−g(y)=ℓ+−u′(μ)2<0.\lim_{y\to-\infty}g(y)=\frac{\ell_--u'(\mu)}{2}>0,\qquad \lim_{y\to\mu^-}g(y)=\frac{\ell_+-u'(\mu)}{2}<0 .y→−∞lim​g(y)=2ℓ−​−u′(μ)​>0,y→μ−lim​g(y)=2ℓ+​−u′(μ)​<0.
  1. A zero of ggg (p. 110). There is a<μa<\mua<μ with g(a)=0g(a)=0g(a)=0; with b=μ+σ2/(μ−a)b=\mu+\sigma^2/(\mu-a)b=μ+σ2/(μ−a) one has a<ba<ba<b, (b−μ)(μ−a)=σ2(b-\mu)(\mu-a)=\sigma^2(b−μ)(μ−a)=σ2 and u(b)−u(a)b−a=u′(a)+u′(b)2\frac{u(b)-u(a)}{b-a}=\frac{u'(a)+u'(b)}{2}b−au(b)−u(a)​=2u′(a)+u′(b)​.

Significance

The result. Through the paper's Proposition 4, two-point support turns the worst-case expected utility over all laws with mean μ\muμ and variance σ2\sigma^2σ2 into a minimization over a single parameter p∈(0,1)p\in(0,1)p∈(0,1) of pu(μ+(1−p)/p σ)+(1−p)u(μ−p/(1−p) σ)pu(\mu+\sqrt{(1-p)/p}\,\sigma)+(1-p)u(\mu-\sqrt{p/(1-p)}\,\sigma)pu(μ+(1−p)/p​σ)+(1−p)u(μ−p/(1−p)​σ). Combined with the projection property of the paper's Section 2, this gives tractable robust counterparts of multivariate stochastic programs whose objective depends on a linear combination x′Rx'Rx′R of random returns. Proposition 5 is the paper's sufficient condition that places a concrete class of utilities in this regime; it is also closed under adding any quadratic.

Formalizing it. The result is proved on paper; no machine-checked version is known. The formalization produces a checked characterization of two-point support (Lemma 1), a reusable encoding of convex-concave and S-shaped functions, and a proof of Proposition 5. The printed final step of the paper's proof is incomplete (see Difficulty), so a complete formal proof requires an argument the paper does not spell out.

Difficulty

Conditions (a) and (b) of Lemma 1 come from a sign change of ggg on (−∞,μ)(-\infty,\mu)(−∞,μ): the two limits in milestone 4 need a l'Hôpital-type argument and the continuity of a monotone derivative. The central difficulty is condition (c): showing that the quadratic built from a pair (a,b)(a,b)(a,b) lies below uuu on the whole line. The obvious argument takes any zero aaa of ggg and counts the intersections of the linear q′q'q′ with the inverse S-shaped u′u'u′. This fails when u′u'u′ is affine on an interval: a zero of ggg can then produce a quadratic that coincides with uuu on [a,b][a,b][a,b] but crosses above uuu just left of aaa. A proof must therefore choose the zero of ggg, or the pair (a,b)(a,b)(a,b), with care, and the curvature hypotheses are not strict.

Formalization scope

  • Functions are ℝ → ℝ; u′u'u′ is deriv u, and the goal assumes Differentiable ℝ u. Finite limits are Tendsto (deriv u) atBot (𝓝 l) and Tendsto (deriv u) atTop (𝓝 l) for some real l; the one-sided limit at μ\muμ is along 𝓝[<] μ.
  • "Increasing" in Definition 2 is strict (StrictMono); the paper writes "nondecreasing" for the weak notion. Convexity and concavity are ConvexOn/ConcaveOn on the open half-lines Set.Iio x₀, Set.Ioi x₀, as printed.
  • A two-point law is encoded by its mass p∈(0,1)p\in(0,1)p∈(0,1) on aaa, with a<ba<ba<b; this is equivalent to a probability measure on R\mathbb RR with support {a,b}\{a,b\}{a,b}, mean μ\muμ and variance σ2\sigma^2σ2.
  • "Intersects it at two points a,ba,ba,b" is read as q(a)=u(a)q(a)=u(a)q(a)=u(a) and q(b)=u(b)q(b)=u(b)q(b)=u(b) for some a<ba<ba<b (at least two contact points). This is the reading under which Lemma 1 is an equivalence.
  • The two-point support property quantifies over σ>0\sigma>0σ>0: no two-point law has variance 000, so including σ=0\sigma=0σ=0 would make the property false for every uuu.
  • The two-point support property is defined from Definition 1 (supporting quadratic plus feasible law), not from Lemma 1's conditions, so Lemma 1 is not a tautology. Every division by b−ab-ab−a, z−yz-yz−y or μ−y\mu-yμ−y occurs under a<ba<ba<b or y<μy<\muy<μ.

Useful infrastructure: l'Hôpital's rule at infinity (Analysis/Calculus/LHopital), Darboux's theorem for derivatives, the intermediate value theorem, and one-sided limits of monotone functions. The shape definitions of Definition 2 are reusable for S-shaped value functions elsewhere (prospect theory, sigmoidal utilities). Proofs of the milestones, of the goal, and alternative arguments for condition (c) are all welcome. Related platform work: Wasserstein Distributionally Robust Optimization II.

Selected references

  • I. Popescu, Robust Mean-Covariance Solutions for Stochastic Optimization, Operations Research 55(1):98–112, 2007. https://doi.org/10.1287/opre.1060.0353
  • J. R. Birge, J. H. Dulá, Bounding separable recourse functions with limited distribution information, Annals of Operations Research 30:277–298, 1991 (cited in Popescu 2007, reference list)
10 thms5 active usersReviewed
Computational GeometryLinear OptimizationTheoretical Computer Science·Captain: mikedeng1

Linear Programming in Linear Time When the Dimension Is Fixed: Fixed-Dimension LP Feasibility Decided in Linear Time on the Real RAMResearch Paper

Motivation

A linear program asks for a point x∈Rdx\in\mathbb{R}^dx∈Rd minimizing cTxc^TxcTx subject to nnn linear inequalities ∑j=1daijxj≥bi\sum_{j=1}^d a_{ij}x_j\ge b_i∑j=1d​aij​xj​≥bi​. Many problems in computational geometry and statistics are linear programs with few variables and very many constraints: separating two point sets by a line or plane, fitting a line in the Chebyshev (L∞L_\inftyL∞​) norm, finding the smallest disk or ball containing a point set (a related convex problem). For these problems the number of variables ddd is a small constant, and what matters is how the running time grows with nnn.

Nimrod Megiddo showed that for every fixed ddd the problem can be solved in time C(d)⋅nC(d)\cdot nC(d)⋅n (J. ACM 31(1), 1984).

Timeline.

  • 1983. Megiddo (SIAM J. Comput. 12) and, independently, Dyer (SIAM J. Comput. 13 (1984)) give linear-time algorithms for d=2d=2d=2 and d=3d=3d=3.
  • 1984. Megiddo extends the method to every fixed ddd, with C(d)<22d+2C(d)<2^{2^{d+2}}C(d)<22d+2 (the paper formalized here).
  • 1988–1991. Clarkson (J. ACM 42 (1995), conference version 1988) gives a randomized algorithm with expected time O(d2n)+dO(d)log⁡nO(d^2n)+d^{O(\sqrt d)}\log nO(d2n)+dO(d​)logn. Seidel (Discrete Comput. Geom. 6 (1991)) gives a simple randomized O(d! n)O(d!\,n)O(d!n) algorithm.
  • 1992–1996. Matoušek, Sharir and Welzl and, independently, Kalai give subexponential randomized bounds. Chazelle and Matoušek derandomize the linear dependence with C(d)=dO(d)C(d)=d^{O(d)}C(d)=dO(d) (J. Algorithms 21 (1996)).

Setting

Fix ddd. An instance is a matrix A∈Rn×dA\in\mathbb{R}^{n\times d}A∈Rn×d and a vector b∈Rnb\in\mathbb{R}^nb∈Rn, and its feasible region is the polyhedron P(A,b)={x∈Rd:Ax≥b}P(A,b)=\{x\in\mathbb{R}^d: Ax\ge b\}P(A,b)={x∈Rd:Ax≥b}. Here nnn is the number of constraints and ddd the number of variables.

The model of computation is the real RAM. A program is a finite list of instructions acting on real registers, integer pointer registers and a memory Z→R\mathbb{Z}\to\mathbb{R}Z→R. It performs exact +,−,×,/+,-,\times,/+,−,×,/ on reals at unit cost, tests the sign of a real, sets, copies, increments, decrements and compares pointers, and loads and stores through pointers. The input is the standard encoding of (A,b)(A,b)(A,b) in memory: the numbers nnn and ddd, then AAA row by row, then bbb. A program decides an instance within TTT steps with output β∈{accept,reject}\beta\in\{\text{accept},\text{reject}\}β∈{accept,reject} if it halts on that output after at most TTT steps.

Megiddo's method rests on multidimensional search. There is an unknown point x∗∈Rdx^*\in\mathbb{R}^dx∗∈Rd and an oracle that, for any hyperplane {x:aTx=b}\{x: a^Tx=b\}{x:aTx=b}, answers whether aTx∗<ba^Tx^*<baTx∗<b, =b=b=b or >b>b>b. Given hyperplanes Hi={aiTx=bi}H_i=\{a_i^Tx=b_i\}Hi​={aiT​x=bi​} with ai≠0a_i\ne0ai​=0, the question is how many oracle calls determine the position of x∗x^*x∗ relative to all of them. A search strategy is a ternary decision tree: inner nodes are hyperplane queries, leaves carry outputs, and the tree is built from the data alone. For linear programming, x∗x^*x∗ is an optimal solution, or a minimizer of the infeasibility function f(x)=max⁡i(bi−aiTx)f(x)=\max_i(b_i-a_i^Tx)f(x)=maxi​(bi​−aiT​x) when the system is infeasible. The oracle is implemented by solving problems in d−1d-1d−1 variables.

Formalization targets

Goal: linear-time feasibility on the real RAM

∀d ∃R ∃C ∀n ∀A∈Rn×d, b∈Rn:R decides within C (n+1) steps whether {x:Ax≥b}≠∅.\forall d\ \exists R\ \exists C\ \forall n\ \forall A\in\mathbb{R}^{n\times d},\,b\in\mathbb{R}^n:\quad R\text{ decides within }C\,(n+1)\text{ steps whether } \{x: Ax\ge b\}\neq\emptyset.∀d ∃R ∃C ∀n ∀A∈Rn×d,b∈Rn:R decides within C(n+1) steps whether {x:Ax≥b}=∅.

The program and the constant depend on ddd only. No explicit form of C(d)C(d)C(d) is fixed.

Milestones

  1. One query settles half of nnn hyperplanes on the line (A(1)=1A(1)=1A(1)=1, B(1)=12B(1)=\tfrac12B(1)=21​).
  2. v(ϵ)=(1,ϵ,…,ϵd−1)v(\epsilon)=(1,\epsilon,\dots,\epsilon^{d-1})v(ϵ)=(1,ϵ,…,ϵd−1) is orthogonal to some aia_iai​ for at most n(d−1)n(d-1)n(d−1) values of ϵ\epsilonϵ, so there is a basis in which all aij≠0a_{ij}\ne0aij​=0.
  3. For hyperplanes of opposite slopes in the (x1,x2)(x_1,x_2)(x1​,x2​) plane, the answers for Hik(1)H^{(1)}_{ik}Hik(1)​ and Hik(2)H^{(2)}_{ik}Hik(2)​ settle one of HiH_iHi​, HkH_kHk​.
  4. A linearly dependent pair of opposite slopes has ai1=ak1=0a_{i1}=a_{k1}=0ai1​=ak1​=0, and the middle hyperplane settles one of them.
  5. Approach I: 2d−12^{d-1}2d−1 queries settle at least ⌊21−2dn⌋\lfloor 2^{1-2^d}n\rfloor⌊21−2dn⌋ hyperplanes.
  6. C(d)log⁡nC(d)\log nC(d)logn queries settle all nnn hyperplanes.
  7. If a hyperplane contains no optimal point, all optimal points lie on one side of it.
  8. The oracle, Case I: at an optimum relative to {xd=0}\{x_d=0\}{xd​=0}, two auxiliary systems decide the side or certify global optimality.
  9. The oracle, Case II: at a minimizer of fff on {xd=0}\{x_d=0\}{xd​=0}, systems (1) and (2) decide the side or certify infeasibility.

Significance

The result. For every fixed dimension, linear programming is solvable in time linear in the number of constraints. The algorithm is also strongly polynomial in fixed dimension: its operation count does not depend on the bit size of the data. Deciding whether the optimum is at most ttt is feasibility of Ax≥bAx\ge bAx≥b together with −cTx≥−t-c^Tx\ge-t−cTx≥−t, so the goal also covers the decision form of optimization. The prune-and-search technique of the paper, which discards a constant fraction of the constraints per round, became a standard tool of computational geometry.

Formalizing it. The result is proved and classical. The platform already has the cases d=1d=1d=1 (linear time) and d=2d=2d=2 (quadratic time, by Fourier–Motzkin elimination) on the same machine and input encoding (SmaleNinth.real_ram_decides_one_variable_lp_linear, SmaleNinth.real_ram_decides_two_variable_lp_quadratic). No machine-checked proof of the general statement is known. The work consists of the query-complexity layer (milestones 1–6), the convex-analytic correctness of the oracle (milestones 7–9), and a real-RAM implementation with a step count linear in nnn, including linear-time median selection. Alternative proofs, for example through Clarkson's or Seidel's algorithms made deterministic, are welcome for the goal.

Difficulty

The obvious approach is to find the optimum by testing constraints one by one or by eliminating variables. Fourier–Motzkin elimination produces Θ(n2)\Theta(n^2)Θ(n2) constraints after one step. Pivoting methods have no known bound linear in nnn. The key difficulty is to discard a constant fraction of the constraints using only a constant number of recursive calls in dimension d−1d-1d−1, when no single hyperplane test gives information about more than one constraint. The multidimensional search layer gives this, and it is where the pairing of hyperplanes by slope and the degenerate cases (dependent pairs, zero coefficients) have to be handled exactly. At the machine level, the step count must stay linear in nnn for a fixed program, so every median selection and every recursive call must be implemented within the budget, with the recursion depth depending on ddd only.

Formalization scope

  • Machine and input. The machine is the platform's real RAM SmaleNinth.RAMProgram with RAMDecidesInTime, and the input convention is SmaleNinth.encodeLP (published definitions, reused unchanged). No instruction is added: there is no LP, median, floor or sort primitive. Time is the number of machine steps.
  • Quantifier order. ∀d ∃R ∃C ∀n,A,b\forall d\ \exists R\ \exists C\ \forall n, A, b∀d ∃R ∃C ∀n,A,b. The bound is C(n+1)C(n+1)C(n+1) in the number nnn of constraints, so that the machine can halt at n=0n=0n=0. The paper's C(d)<22d+2C(d)<2^{2^{d+2}}C(d)<22d+2 counts unspecified units of "effort" with an unquantified θ(nd)\theta(nd)θ(nd) term, and it is not transferred to machine steps. Where a milestone's proof fixes a constant exactly, the constant is stated: 2d−12^{d-1}2d−1 queries and ⌊n/22d−1⌋\lfloor n/2^{2^d-1}\rfloor⌊n/22d−1⌋ settled hyperplanes in milestone 5.
  • Feasibility only. The machine outputs accept or reject. Returning an optimizer, "unbounded", or a minimizer of fff is not part of the goal. The case d=0d=0d=0 is included.
  • Query trees. Nodes are queries compare (a ⬝ᵥ x) b and nothing else, leaves hold fixed values, and correctness is required for every xxx. A tree over arbitrary tests of xxx would make milestones 5 and 6 empty, and it is excluded by the definition.
  • Indices. The paper's x1,x2x_1,x_2x1​,x2​ are indices 0, 1 of Fin (d + 2), and its xdx_dxd​ is Fin.last d of Fin (d + 1).
  • Corrections. Two passages of §4 are stated in corrected form. The Case I auxiliary objective includes the ±cd\pm c_d±cd​ term of the direction. In Case II, feasibility of (1) puts improvement in {xd>0}\{x_d>0\}{xd​>0}, where the page's last sentence says {xd<0}\{x_d<0\}{xd​<0}. The pairing claim carries ak1ai2−ak2ai1≠0a_{k1}a_{i2}-a_{k2}a_{i1}\ne0ak1​ai2​−ak2​ai1​=0, the hypothesis its argument uses, since linear independence alone does not give it.
  • Not included. Approach II and its bound O(n(log⁡n)d2)O(n(\log n)^{d^2})O(n(logn)d2), the remarks on slowly growing ddd, the randomized variants, and the applications of §1.
  • Reusable parts. The query-tree definition and milestones 1–6 apply to any prune-and-search problem with a hyperplane oracle. The oracle lemmas (7–9) are statements about convex piecewise-linear functions and polyhedra.

Selected references

  • N. Megiddo, Linear programming in linear time when the dimension is fixed, J. ACM 31(1):114–127, 1984. https://doi.org/10.1145/2422.322418
  • N. Megiddo, Linear-time algorithms for linear programming in R3R^3R3 and related problems, SIAM J. Comput. 12(4):759–776, 1983. https://doi.org/10.1137/0212052
  • M. E. Dyer, Linear time algorithms for two- and three-variable linear programs, SIAM J. Comput. 13(1):31–45, 1984. https://doi.org/10.1137/0213003
  • K. L. Clarkson, Las Vegas algorithms for linear and integer programming when the dimension is small, J. ACM 42(2):488–499, 1995. https://doi.org/10.1145/201019.201036
  • R. Seidel, Small-dimensional linear programming and convex hulls made easy, Discrete Comput. Geom. 6:423–434, 1991. https://doi.org/10.1007/BF02574699
  • B. Chazelle, J. Matoušek, On linear-time deterministic algorithms for optimization problems in fixed dimension, J. Algorithms 21(3):579–597, 1996. https://doi.org/10.1006/jagm.1996.0046
16 thms5 active usersReviewed
🏆Completed
Convex OptimizationMachine LearningOptimization·Captain: mikedeng1

Logarithmic Regret Algorithms for Online Convex Optimization 1: Logarithmic Regret of Online Gradient DescentResearch Paper

Motivation

Online convex optimization is a repeated game between a learner and an adversary. In each round t=1,…,Tt=1,\dots,Tt=1,…,T the learner commits to a point xtx_txt​ of a convex set P⊆Rn\mathcal P\subseteq\mathbb R^nP⊆Rn; only then is a convex cost function ftf_tft​ revealed, and the learner pays ft(xt)f_t(x_t)ft​(xt​). The learner is judged by its regret: its total cost minus the total cost of the best fixed point chosen in hindsight. The model covers online portfolio selection, online regression, prediction with expert advice and the analysis of stochastic gradient methods, and it is the standard language of online learning theory.

Zinkevich (ICML 2003) showed that projected gradient descent with step sizes of order 1/t1/\sqrt t1/t​ has regret O(GDT)O(GD\sqrt T)O(GDT​) for any convex costs with gradients bounded by GGG on a set of diameter DDD, and this rate cannot be improved for linear costs. Hazan, Agarwal and Kale (Machine Learning 69, 2007) asked what curvature buys. Their first result, the subject of this mission, is that the same algorithm with the faster step sizes 1/(Ht)1/(Ht)1/(Ht) has regret only logarithmic in TTT once every cost function is HHH-strongly convex. The paper's other three results (the Online Newton Step, Follow the Approximate Leader and Exponentially Weighted Online Optimization, for exp-concave costs) are the subject of the other missions of this series.

Setting

Fix n∈Nn\in\mathbb Nn∈N and a nonempty, closed, bounded, convex set P⊆Rn\mathcal P\subseteq\mathbb R^nP⊆Rn, with the Euclidean norm ∥⋅∥2\|\cdot\|_2∥⋅∥2​ (§2.1, p. 171).

  • Cost functions. A sequence f1,f2,⋯:Rn→Rf_1,f_2,\dots:\mathbb R^n\to\mathbb Rf1​,f2​,⋯:Rn→R. Write ∇ft(x)\nabla f_t(x)∇ft​(x) for the gradient and ∇2ft(x)\nabla^2 f_t(x)∇2ft​(x) for the Hessian.
  • Gradient bound. A number GGG with ∥∇ft(x)∥2≤G\|\nabla f_t(x)\|_2\le G∥∇ft​(x)∥2​≤G for all x∈Px\in\mathcal Px∈P and all rounds ttt (p. 172).
  • HHH-strong convexity (p. 172). For H>0H>0H>0, fff is HHH-strongly convex on P\mathcal PP when it is twice differentiable and ∇2f(x)⪰HIn\nabla^2 f(x)\succeq H I_n∇2f(x)⪰HIn​ for every x∈Px\in\mathcal Px∈P, i.e. v⊤∇2f(x)v≥H∥v∥22v^\top\nabla^2 f(x)v\ge H\|v\|_2^2v⊤∇2f(x)v≥H∥v∥22​ for all vvv.
  • Euclidean projection. ΠP(y)\Pi_{\mathcal P}(y)ΠP​(y) is the point of P\mathcal PP nearest to yyy, ΠP(y)=arg⁡min⁡x∈P∥x−y∥2\Pi_{\mathcal P}(y)=\arg\min_{x\in\mathcal P}\|x-y\|_2ΠP​(y)=argminx∈P​∥x−y∥2​ (IsProj P y z).
  • Online Gradient Descent (Fig. 1, p. 174), with step sizes η1,η2,…\eta_1,\eta_2,\dotsη1​,η2​,…: x1∈Px_1\in\mathcal Px1​∈P is arbitrary, and in iteration t>1t>1t>1
xt=ΠP(xt−1−ηt∇ft−1(xt−1))x_t=\Pi_{\mathcal P}\bigl(x_{t-1}-\eta_t\nabla f_{t-1}(x_{t-1})\bigr)xt​=ΠP​(xt−1​−ηt​∇ft−1​(xt−1​))

(IsOGDRun P η f x).

  • Regret (p. 171): RegretT=∑t=1Tft(xt)−min⁡x∈P∑t=1Tft(x)\mathrm{Regret}_T=\sum_{t=1}^T f_t(x_t)-\min_{x\in\mathcal P}\sum_{t=1}^T f_t(x)RegretT​=∑t=1T​ft​(xt​)−minx∈P​∑t=1T​ft​(x), and RegretT(OGD)\mathrm{Regret}_T(\mathrm{OGD})RegretT​(OGD) is its supremum over all cost sequences.

Formalization targets

Goal: Theorem 1 (p. 175)

For H>0H>0H>0, cost functions that are HHH-strongly convex on P\mathcal PP with gradients bounded by GGG on P\mathcal PP, and any run of Online Gradient Descent whose step after round ttt is ηt+1=1/(Ht)\eta_{t+1}=1/(Ht)ηt+1​=1/(Ht), for every T≥1T\ge1T≥1 and every u∈Pu\in\mathcal Pu∈P:

∑t=1T(ft(xt)−ft(u)) ≤ G22H (1+log⁡T).\sum_{t=1}^{T}\bigl(f_t(x_t)-f_t(u)\bigr)\ \le\ \frac{G^2}{2H}\,\bigl(1+\log T\bigr).t=1∑T​(ft​(xt​)−ft​(u)) ≤ 2HG2​(1+logT).

This is LogRegretOCO.OGD.ogd_regret_bound. The constant is the paper's, and at T=1T=1T=1 the bound reads G2/(2H)G^2/(2H)G2/(2H).

Milestones

  1. Eq. (1), the strong-convexity inequality: for x,y∈Px,y\in\mathcal Px,y∈P, 2(f(x)−f(y))≤2∇f(x)⊤(x−y)−H∥y−x∥222(f(x)-f(y))\le 2\nabla f(x)^\top(x-y)-H\|y-x\|_2^22(f(x)−f(y))≤2∇f(x)⊤(x−y)−H∥y−x∥22​.
  2. Lemma 8 with A=InA=I_nA=In​: for a convex P\mathcal PP, z=ΠP(y)z=\Pi_{\mathcal P}(y)z=ΠP​(y) and a∈Pa\in\mathcal Pa∈P, ∥y−a∥22≥∥z−a∥22\|y-a\|_2^2\ge\|z-a\|_2^2∥y−a∥22​≥∥z−a∥22​. This is already on the platform, proved, as UnderstandingML.projection_lemma, and is reused as a reference item.
  3. Eq. (2), the one-step inequality: for z=ΠP(x−ηg)z=\Pi_{\mathcal P}(x-\eta g)z=ΠP​(x−ηg), η>0\eta>0η>0, ∥g∥2≤G\|g\|_2\le G∥g∥2​≤G and u∈Pu\in\mathcal Pu∈P, 2g⊤(x−u)≤(∥x−u∥22−∥z−u∥22)/η+ηG22g^\top(x-u)\le(\|x-u\|_2^2-\|z-u\|_2^2)/\eta+\eta G^22g⊤(x−u)≤(∥x−u∥22​−∥z−u∥22​)/η+ηG2.

The last step of the argument is the harmonic-sum bound ∑t=1T1/t≤1+log⁡T\sum_{t=1}^T 1/t\le 1+\log T∑t=1T​1/t≤1+logT, which Mathlib already provides (harmonic_le_one_add_log).

Significance

The result. Theorem 1 separates two regimes of online convex optimization: Θ(T)\Theta(\sqrt T)Θ(T​) regret for general convex costs and O(log⁡T)O(\log T)O(logT) for strongly convex ones, with an algorithm that costs one gradient and one projection per round. Through the online-to-batch conversion, the same step-size schedule gives the O(log⁡T/T)O(\log T/T)O(logT/T) rate of stochastic gradient descent on strongly convex objectives. The theorem is also the reference point for the paper's weaker exp-concavity assumption, under which the other three algorithms obtain O(nlog⁡T)O(n\log T)O(nlogT) regret.

Formalizing it. The result is proved and well known; what this mission adds is a machine-checked proof with the paper's exact constant. The Prove2Me platform holds a statement of the textbook version of this theorem (Hazan, Introduction to Online Convex Optimization, Theorem 3.3), OnlineConvexOpt.FirstOrder.online_gradient_descent_strongly_convex_regret, but it is Disproved: its regret is written with a real infimum over the decision set, which Lean evaluates to a junk value. No proved version of the logarithmic bound is on the platform. The strong-convexity inequality, the one-step projected-gradient inequality and the telescoping argument are reusable by every later formalization of gradient methods on the platform.

Difficulty

The argument is short, and its difficulty lies in the bookkeeping. The telescoping sum of squared distances cancels exactly only with the right step-size indexing: the step taken after round ttt must be 1/(Ht)1/(Ht)1/(Ht). Reading Fig. 1 literally with ηt=1/(Ht)\eta_t=1/(Ht)ηt​=1/(Ht) makes the step after round ttt equal to 1/(H(t+1))1/(H(t+1))1/(H(t+1)), and the sum then leaves an uncancelled term H2∥x1−x∗∥22\frac H2\|x_1-x^*\|_2^22H​∥x1​−x∗∥22​ that the printed bound does not contain. Strong convexity is a statement about the Hessian, so the curvature inequality (1) needs a second-order Taylor expansion along a segment of P\mathcal PP; convexity of P\mathcal PP keeps the segment inside the set where the Hessian bound holds. The projection step needs the obtuse-angle property of Euclidean projection onto a convex set.

Formalization scope

Points are EuclideanSpace ℝ (Fin n), so ∥⋅∥\|\cdot\|∥⋅∥ is the Euclidean norm (the sup norm of Fin n → ℝ would change the gradient bound). Cost functions are functions on all of Rn\mathbb R^nRn, as the paper's use of gradients and Hessians presupposes; the gradient is Mathlib's gradient, and the Hessian quadratic form is the second Fréchet derivative applied to (v,v)(v,v)(v,v). Rounds are 1-based: x 0, f 0 and η 1 are never read, and sums run over Finset.Icc 1 T. The run of the algorithm is a predicate on the whole trajectory, required at every round; the projection is a predicate (nearest point of P\mathcal PP), which is unique for nonempty closed convex P\mathcal PP.

Choices and corrections relative to the printed text:

  • Step-size index. Theorem 1 prints "step sizes ηt=1Ht\eta_t=\frac1{Ht}ηt​=Ht1​", while its proof sets ηt+1=1/(Ht)\eta_{t+1}=1/(Ht)ηt+1​=1/(Ht) for the step after round ttt. The goal uses the proof's indexing, as a hypothesis η (t + 1) = 1 / (H * t) for t≥1t\ge1t≥1 on the step sizes of Fig. 1.
  • Eq. (2). The paper prints "5∇t⊤(xt−x∗)5\nabla_t^\top(x_t-x^*)5∇t⊤​(xt​−x∗)"; the 555 is a typo for 222, and the milestone states 222. The verbatim milestone text keeps the printed 555.
  • Regret. Regret is stated against every comparator u∈Pu\in\mathcal Pu∈P; the minimum over the compact set P\mathcal PP is attained, so this is the paper's statement. The expectation in the paper's regret is vacuous for this deterministic algorithm, and the supremum over cost sequences is the universal quantifier over fff.
  • Hypotheses. Strong convexity and the gradient bound are required for the rounds 1,…,T1,\dots,T1,…,T only. Convexity of each ftf_tft​, a standing assumption of §2.2, follows from HHH-strong convexity on the convex set P\mathcal PP and is not added. No diameter bound enters Theorem 1.

A regret bound written with a real ⨅/sInf over P\mathcal PP, or a bound for an arbitrary sequence satisfying Eq. (2) instead of a run of the paper's algorithm, would not be Theorem 1; the goal quantifies over every comparator in P\mathcal PP and carries the run predicate, the Hessian hypothesis and the gradient bound. The hypotheses are jointly satisfiable: on the closed unit ball, ft(x)=H2∥x∥22f_t(x)=\frac H2\|x\|_2^2ft​(x)=2H​∥x∥22​ with G=HG=HG=H meets all of them.

Contributions welcome: proofs of the two inequalities and of the goal; a general projection lemma for positive semidefinite AAA (Lemma 8 in full, needed by the Online Newton Step mission) is reusable beyond this mission.

Selected references

  • E. Hazan, A. Agarwal, S. Kale, Logarithmic regret algorithms for online convex optimization, Machine Learning 69 (2007), 169–192. https://doi.org/10.1007/s10994-007-5016-8
  • M. Zinkevich, Online convex programming and generalized infinitesimal gradient ascent, ICML 2003. https://www.aaai.org/Papers/ICML/2003/ICML03-120.pdf
  • E. Hazan, Introduction to Online Convex Optimization, 2nd ed., MIT Press 2022 (Theorem 3.3). https://arxiv.org/abs/1909.05207
  • S. Shalev-Shwartz, S. Ben-David, Understanding Machine Learning: From Theory to Algorithms, Cambridge University Press 2014 (Lemma 14.9, the projection lemma). https://doi.org/10.1017/CBO9781107298019
5 thms5 active usersReviewed
🏆Completed
Bandit AlgorithmsOptimizationProbability·Captain: naimengye

Multi-armed Bandit Allocation Indices II: Jobs, Parallel Machines, Search and Bandit-Dependent DiscountingTextbook

Motivation

Chapter 3 of Gittins, Glazebrook and Weber, Multi-armed Bandit Allocation Indices (2nd ed., doi:10.1002/9780470980033), asks which of the assumptions behind the index theorem can be dropped and which cannot, using the simplest bandit processes there are: jobs, which pay a single reward when they complete. Along the way it settles three questions that matter on their own. When identical machines run in parallel, which schedule of deterministic jobs minimizes the total completion time (Baker's theorem), and what does the equal-ratio case look like (Theorem 3.3)? When an object is hidden in one of several boxes and each search costs money and may fail, in what order should the boxes be searched (Blackwell's search-index theorem, Theorem 3.6)? And when two bandit processes discount at different rates, is there still an index policy (Nash's Theorem 3.4)? The search theorem is the chapter's example of what the index machinery yields once undiscounted criteria are admitted through the limit γ↓0\gamma \downarrow 0γ↓0; the scheduling results are the source of the counterexamples that show a second machine breaks the index structure.

Setting

Jobs on parallel machines. There are nnn jobs with service times si>0s_i > 0si​>0 and weights cic_ici​, and mmm identical machines. A schedule assigns each job a machine and a position in that machine's processing order; each machine processes its jobs consecutively. The completion time CiC_iCi​ of a job is the total service time of the jobs on its machine up to and including it; the flow time is ∑iCi\sum_i C_i∑i​Ci​ and the weighted flow time ∑iciCi\sum_i c_i C_i∑i​ci​Ci​. The load of a machine is the total service time assigned to it, and δj\delta_jδj​ is its excess over the average S/mS/mS/m. The SPT list schedule sorts the jobs by increasing service time and deals them out cyclically to the machines. The level of a job is its position counted from the end of its machine's schedule.

The search problem (Problem 3). A stationary object is hidden in one of nnn boxes, in box iii with probability pip_ipi​. A search of box iii costs cic_ici​ and finds the object with probability qiq_iqi​ if it is there. A search policy is a sequence of boxes; after NiN_iNi​ unsuccessful searches of box iii the posterior probability that the object is there is proportional to pi(1−qi)Nip_i(1-q_i)^{N_i}pi​(1−qi​)Ni​, and the search index of the box is pi′qi/cip'_i q_i/c_ipi′​qi​/ci​, the current probability of finding the object per unit cost. The expected cost of a policy is ∑kcσkPr⁡[not found by the first k searches]\sum_k c_{\sigma_k}\Pr[\text{not found by the first } k \text{ searches}]∑k​cσk​​Pr[not found by the first k searches].

Two discount factors. Bandit process AAA (state space SAS_ASA​, kernel PAP_APA​, reward rAr_ArA​) discounts at rate aaa and BBB at rate bbb; a reward obtained at time ttt is worth ata^tat or btb^tbt times its face value according to which process produced it. The cross indices are νAB(x)=sup⁡τ>0E∑t<τatrA(x(t))/E[1−bτ]\nu_{AB}(x) = \sup_{\tau>0} \mathbb{E}\sum_{t<\tau} a^t r_A(x(t)) \big/ \mathbb{E}[1 - b^\tau]νAB​(x)=supτ>0​E∑t<τ​atrA​(x(t))/E[1−bτ] and νBA(y)\nu_{BA}(y)νBA​(y) with the roles exchanged: each is computed from one process's stopping times but with the other's discount factor in the denominator. The book prints the denominator as E∫0τbt dt\mathbb{E}\int_0^\tau b^t\,dtE∫0τ​btdt and compares νAB(x)\nu_{AB}(x)νAB​(x) with νBA(y)\nu_{BA}(y)νBA​(y) at every time; both are corrected here (see the Theorem 3.4 item), since the printed rule is not optimal. The family is the two-armed Markov bandit of the Bandit Algorithms model on the disjoint union SA⊕SBS_A \oplus S_BSA​⊕SB​.

Formalization targets

Goal: Theorem 3.6

For prior probabilities pi≥0p_i \ge 0pi​≥0 summing to one, detection probabilities 0<qi≤10 < q_i \le 10<qi​≤1 and costs ci>0c_i > 0ci​>0, a search policy is optimal if and only if at every step it searches a box of maximal current index pi′qi/cip'_i q_i / c_ipi′​qi​/ci​:

optimal(σ)  ⟺  ∀k, ∀i: pi(1−qi)Ni(k)qici≤pσk(1−qσk)Nσk(k)qσkcσk.\text{optimal}(\sigma) \iff \forall k,\ \forall i:\ \frac{p_i (1-q_i)^{N_i(k)} q_i}{c_i} \le \frac{p_{\sigma_k}(1-q_{\sigma_k})^{N_{\sigma_k}(k)} q_{\sigma_k}}{c_{\sigma_k}}.optimal(σ)⟺∀k, ∀i: ci​pi​(1−qi​)Ni​(k)qi​​≤cσk​​pσk​​(1−qσk​​)Nσk​​(k)qσk​​​.

Milestones

Theorem 3.3, as the identity ∑iciCi=κ2(∑isi2+S2/m+∑jδj2)\sum_i c_i C_i = \frac{\kappa}{2}\big(\sum_i s_i^2 + S^2/m + \sum_j \delta_j^2\big)∑i​ci​Ci​=2κ​(∑i​si2​+S2/m+∑j​δj2​) when ci=κsic_i = \kappa s_ici​=κsi​; Theorem 3.4 in discrete time and corrected, that a policy selecting, at time ttt, AAA when atνAB(x)>btνBA(y)a^t \nu_{AB}(x) > b^t \nu_{BA}(y)atνAB​(x)>btνBA​(y) and BBB when atνAB(x)<btνBA(y)a^t \nu_{AB}(x) < b^t \nu_{BA}(y)atνAB​(x)<btνBA​(y) attains the supremum of the bandit-dependently discounted payoff; Theorem 3.7, that SPT minimizes the flow time on mmm machines and that the optimal schedules are exactly those placing the rrr-th block of mmm longest jobs at level rrr from the end, for some order of the equal jobs.

Significance

Theorem 3.6 is the classical solution of the discrete search problem (Blackwell, reported by Matula 1964; Kadane 1969): the greedy rule in probability-per-cost is optimal, and every optimal policy is of that form. The book derives it from the index theorem through an auxiliary family of bandit processes (Problem 3A) and the undiscounted limit (Corollary 3.5), which is why it sits in this chapter; as a statement it is elementary and self-contained, and it is the template for the tax problems and Klimov's model of Chapter 4. Theorem 3.7 and Theorem 3.3 are the positive results about parallel machines that survive the loss of the index structure; Theorem 3.4 shows the index theorem's shape persisting under bandit-dependent discounting, with two twists: each index depends on the other process's discount factor, which is exactly why it does not extend to three processes, and the comparison at time ttt weighs the indices by ata^tat and btb^tbt, so the rule is not stationary in the states. The printed statement misses the second twist and misnormalizes the first; two one-state bandits paying 0.180.180.18 (a=0.9a = 0.9a=0.9) and 111 (b=0.5b = 0.5b=0.5) are best played B,B,BB, B, BB,B,B and then AAA forever, which no stationary rule does.

None of these is machine-checked. Formalizing them gives the platform an optimality theorem for an infinite-horizon search process with an explicit index characterization in both directions, the standard parallel-machine flow-time results with a precise uniqueness clause, and a first statement on the Bandit Algorithms two-armed model with unequal discounting.

Difficulty

For Theorem 3.6 the obvious route, comparing two adjacent searches, gives only that interchanging a pair in the wrong index order lowers the cost; turning that into optimality over all infinite sequences needs that an optimal policy exists (costs are bounded below by zero and the index policy has finite cost), that a policy neglecting a box of positive prior has infinite cost, and that any first deviation from the index rule can be improved by moving a later search forward, which requires tracking how the not-found probability changes along the whole tail. The converse direction is the same interchange run backwards, and the tie case must be handled so that the equivalence is exact. Theorem 3.7's first part follows from writing the flow time as ∑iℓisi\sum_i \ell_i s_i∑i​ℓi​si​ with ℓi\ell_iℓi​ the level and applying a rearrangement inequality over the multiset of levels, but the multiset of levels itself depends on how many jobs each machine gets, so balancing the machine counts is part of the argument; the uniqueness clause needs both that the multiset is forced and that the pairing of levels with service times is forced by strict monotonicity. Theorem 3.4 is the hardest: the book's proof changes the time scale of each process so that the two share a discount factor, which produces a semi-Markov family, and applies the index theorem there; in discrete time on the Markov model one needs either a discrete analogue of that argument or a direct prevailing-charge proof with two charge scales.

Formalization scope

Schedules are assignments of machines and positions with distinct positions on a machine, without idling; the flow-time quantities are finite sums over Fin n. The SPT schedule is defined by the ascending rank of a job (ties broken by index) and carries its own injectivity proof. The search cost is a series in [0,∞][0, \infty][0,∞] of nonnegative terms, so no summability hypothesis is needed and "infinite cost" is literal; the index is stated unnormalized, pi(1−qi)Niqi/cip_i(1-q_i)^{N_i} q_i/c_ipi​(1−qi​)Ni​qi​/ci​, which orders the boxes exactly as the posterior index does. The two-discount family uses the platform's MarkovBanditPolicy 2 (S_A ⊕ S_B) and markovBanditMeasure with the kernel that moves an AAA-state by PAP_APA​ and a BBB-state by PBP_BPB​; the payoff is the round-by-round series ∑t(atE[r1{At=A}]+btE[r1{At=B}])\sum_t (a^t \mathbb{E}[r\mathbf 1\{A_t = A\}] + b^t \mathbb{E}[r\mathbf 1\{A_t = B\}])∑t​(atE[r1{At​=A}]+btE[r1{At​=B}]), absolutely summable for bounded rewards, and the policy condition is imposed at histories whose current states have the right types (all reachable histories do). Hypotheses: m≥1m \ge 1m≥1, service times positive; pi≥0p_i \ge 0pi​≥0, ∑pi=1\sum p_i = 1∑pi​=1, 0<qi≤10 < q_i \le 10<qi​≤1, ci>0c_i > 0ci​>0; countable state spaces, bounded rewards, a,b∈(0,1)a, b \in (0,1)a,b∈(0,1).

Trivializing readings are excluded: the search "iff" is over all sequences, not a finite horizon; the uniqueness clause is stated in full, with ties resolved by any ranking of the equal jobs; the cross indices are suprema over positive stopping times (denominators at least one), and the two-discount rule weighs the indices by ata^tat, btb^tbt. Welcome contributions: the rearrangement and level-count lemmas for schedules, the interchange lemma for the search cost, and a discrete prevailing-charge argument for Theorem 3.4.

Selected references

  • J. Gittins, K. Glazebrook, R. Weber, Multi-armed Bandit Allocation Indices, 2nd ed., Wiley, 2011, Chapter 3. doi:10.1002/9780470980033
  • D. W. Matula, A periodic optimal search, American Mathematical Monthly 71(1), 1964. doi:10.2307/2311300
  • J. B. Kadane, Quiz show problems, Journal of Mathematical Analysis and Applications 27(3), 1969. doi:10.1016/0022-247X(69)90140-2
  • K. R. Baker, Introduction to Sequencing and Scheduling, Wiley, 1974.
  • P. Nash, A generalized bandit problem, Journal of the Royal Statistical Society B 42(2), 1980. doi:10.1111/j.2517-6161.1980.tb01119.x
8 thms5 active usersReviewed
🏆Completed
Control TheoryDynamic ProgrammingOptimization·Captain: Shuze Chen

Dynamic Programming and Optimal Control IV: LQR and the Riccati EquationTextbook

Motivation

The discrete-time Riccati equation is the central object of linear-quadratic optimal control — the design equation behind LQR/LQG controllers in every modern control stack. Proposition 4.4.1 of Bertsekas, Dynamic Programming and Optimal Control, Vol. I (3rd ed., 2005, §4.1) packages its asymptotic theory: under controllability and observability the Riccati iteration converges to the unique positive semidefinite solution of the algebraic Riccati equation and the resulting closed loop is stable. Alongside it, Lemma 4.2.1 of §4.2 develops K-convexity, the analytical engine behind Scarf's optimality of (s,S)(s,S)(s,S) inventory policies — a foundational result of operations research. Neither the Riccati asymptotics nor K-convexity exists in Mathlib.

Setting

Matrices A∈Rn×nA \in \mathbb{R}^{n\times n}A∈Rn×n, B∈Rn×mB \in \mathbb{R}^{n\times m}B∈Rn×m, Q=C⊤C⪰0Q = C^\top C \succeq 0Q=C⊤C⪰0, R≻0R \succ 0R≻0. The Riccati operator (BertsekasRiccatiMap)

F(P)=A⊤(P−PB(B⊤PB+R)−1B⊤P)A+Q.F(P) = A^\top\big(P - P B (B^\top P B + R)^{-1} B^\top P\big) A + Q.F(P)=A⊤(P−PB(B⊤PB+R)−1B⊤P)A+Q.

(A,B)(A,B)(A,B) is controllable if [B,AB,…,An−1B][B, AB, \dots, A^{n-1}B][B,AB,…,An−1B] has rank nnn (BertsekasControllablePair); (A,C)(A,C)(A,C) is observable if (A⊤,C⊤)(A^\top, C^\top)(A⊤,C⊤) is controllable (BertsekasObservablePair). Separately, g:R→Rg : \mathbb{R} \to \mathbb{R}g:R→R is KKK-convex (BertsekasKConvex, Def. 4.2.1) if K+g(z+y)≥g(y)+zb(g(y)−g(y−b))K + g(z+y) \ge g(y) + \tfrac{z}{b}(g(y) - g(y-b))K+g(z+y)≥g(y)+bz​(g(y)−g(y−b)) for all z≥0z \ge 0z≥0, b>0b > 0b>0, yyy.

Target

∃ P≻0:F(P)=P,P unique among P′⪰0,Fk(P0)→P  ∀P0⪰0,ρ(A+BL)<1,\exists\, P \succ 0:\quad F(P) = P,\quad P \text{ unique among } P' \succeq 0,\quad F^{k}(P_0) \to P \ \ \forall P_0 \succeq 0,\quad \rho\big(A + BL\big) < 1,∃P≻0:F(P)=P,P unique among P′⪰0,Fk(P0​)→P  ∀P0​⪰0,ρ(A+BL)<1,

with L=−(B⊤PB+R)−1B⊤PAL = -(B^\top P B + R)^{-1} B^\top P AL=−(B⊤PB+R)−1B⊤PA — BertsekasDP.riccati_convergence_stability (goal). Milestones: Lemma 4.2.1(a)–(d) (kconvex_of_convex, kconvex_combination, kconvex_expectation, kconvex_sS_structure), culminating in the (s,S)(s,S)(s,S) structure theorem for continuous coercive KKK-convex functions.

Significance

The Riccati result is the mathematical license behind steady-state LQR design: it guarantees the design equation has one meaningful solution, that iterating the finite-horizon recursion finds it, and that the resulting feedback is stabilizing. Formally it would seed a Mathlib-adjacent theory of matrix fixed-point iterations, positive semidefinite order, and spectral-radius stability. The K-convexity milestones are self-contained real analysis, each of independent reuse value for inventory theory; part (d) is the engine of (s,S)(s,S)(s,S)-policy optimality. All results are classical and proved in the book; the formal work is new.

Difficulty

The Riccati proof interleaves monotonicity of FFF on the psd cone, boundedness from controllability (a steering argument), positivity from observability, and stability extracted from the fixed-point identity via a Lyapunov argument — several pieces of matrix analysis (psd order, congruence, Schur-type manipulations, spectral radius vs. convergence of powers) that must be built or located in Mathlib. The naive route of diagonalizing AAA fails: nothing is symmetric about A+BLA + BLA+BL. For Lemma 4.2.1(d), the difficulty is that ggg is not convex: the minimizer structure must come from the K-convexity inequality applied at carefully chosen points, plus continuity and coercivity.

Formalization scope

Real matrices over Fin n; Matrix.PosSemidef/PosDef; matrix inverse is Mathlib's total inverse (zero on singular input — harmless here since B⊤PB+R≻0B^\top P B + R \succ 0B⊤PB+R≻0 along the relevant iterates, which the proof must establish); convergence in the entrywise topology; eigenvalues via spectrum ℂ of the complexified matrix, all strictly inside the unit circle. Rank-based controllability exactly as Def. 4.1.1. K-convexity is stated for all real KKK; note K≥0K \ge 0K≥0 is forced whenever it is satisfiable (z=0z = 0z=0), and the expectation milestone is stated for finitely supported disturbances (integrability automatic).

Selected references

  • D. P. Bertsekas, Dynamic Programming and Optimal Control, Vol. I, 3rd ed., Athena Scientific, 2005. (Prop. 4.4.1, Def. 4.1.1, §4.2, Lemma 4.2.1.) http://www.athenasc.com/dpbook.html
  • R. E. Kalman, Contributions to the theory of optimal control, Bol. Soc. Mat. Mexicana 5 (1960), 102–119.
  • H. Scarf, The optimality of (S, s) policies in the dynamic inventory problem, in Mathematical Methods in the Social Sciences, Stanford Univ. Press, 1960.
9 thms5 active usersReviewed
🏆Completed
Convex OptimizationOptimization·Captain: Shuze Chen

Convex Optimization III: Conic Duality and the S-procedureTextbook

Two quadratic functions can be compared losslessly. The S-procedure says that, when the constraint is strictly feasible, the implication

q1(x)≤0  ⟹  q2(x)≤0,qk(x)=xTFkx+2gkTx+hk,q_1(x) \le 0 \;\Longrightarrow\; q_2(x) \le 0, \qquad q_k(x) = x^{T}F_k x + 2g_k^{T}x + h_k,q1​(x)≤0⟹q2​(x)≤0,qk​(x)=xTFk​x+2gkT​x+hk​,

holds if and only if a single nonnegative multiplier certifies it as a matrix inequality, λ[F1g1g1Th1]⪰[F2g2g2Th2]\lambda \begin{bmatrix} F_1 & g_1 \\ g_1^{T} & h_1\end{bmatrix} \succeq \begin{bmatrix} F_2 & g_2 \\ g_2^{T} & h_2\end{bmatrix}λ[F1​g1T​​g1​h1​​]⪰[F2​g2T​​g2​h2​​] for some λ≥0\lambda \ge 0λ≥0. It is a cornerstone of control theory, trust-region methods and robust optimization, and a rare case in which a nonconvex problem has zero duality gap. The route runs through the theory this mission builds from Boyd & Vandenberghe §5.8–5.9 and Appendix B: strong alternatives for convex inequality systems, cone-program strong duality under a generalized Slater condition, semidefinite programming duality, the LMI theorems of alternatives, and the hidden convexity of the joint range of two quadratic forms.

17 thms5 active usersReviewed
🏆Completed
Convex OptimizationOptimization·Captain: Shuze Chen

Convex Optimization I: Prékopa's TheoremTextbook

Log-concave functions are the meeting point of convex analysis and probability: densities of Gaussian, exponential, uniform and Wishart distributions are all log-concave, and countless facts of applied probability flow from one structural theorem — integrating out variables preserves log-concavity. This mission builds the convex-analysis spine of Boyd & Vandenberghe's Convex Optimization (Chapters 2–3) — separation and supporting hyperplanes, dual cones, the first- and second-order differential characterizations of convexity, Fenchel conjugacy — and climbs to Prékopa's theorem via the Prékopa–Leindler inequality, a landmark of Brunn–Minkowski theory absent from Mathlib.

29 thms5 active usersReviewed
🏆Completed
Machine LearningQuantum InformationStochastic Systems·Captain: tianyipeng

Markov Entanglement: Value Decomposition Error in Multi-agent MDPsResearch Paper

Value decomposition — approximating the value of a joint state by a sum of per-agent local values — is a staple of multi-agent dynamic programming and reinforcement learning, from index policies for restless bandits to modern MARL architectures, yet it is normally used without justification. Chen and Peng (arXiv:2506.02385) supply one. They show a multi-agent MDP admits an exact value decomposition precisely when its transition matrix is not entangled — a notion built in direct analogy with quantum entanglement — and then turn that qualitative characterisation into a quantitative one: a measure of Markov entanglement bounds the decomposition error in general. This mission formalizes that core theory. The goal is Theorem 6, the general N-agent bound in the occupancy-weighted norm; the milestones are the equivalence between separability and exact decomposition, the perturbation machinery that carries a one-step transition error into a value-function error, and the extensions to shared global state and shared rewards. The paper's restless-bandit application, which needs mean-field machinery of its own, is left to a second mission in the series.

25 thms5 active usersReviewed
🏆Completed
Linear OptimizationOptimization·Captain: Shuze Chen

Introduction to Linear Optimization X: Max-Flow Min-CutTextbook

How much flow can be sent from a source sss to a sink ttt through a network with arc capacities uij∈(0,∞]u_{ij}\in(0,\infty]uij​∈(0,∞] — and what certifies that no more is possible? This mission formalizes §7.4-7.5 of Bertsimas & Tsitsiklis. The circulation calculus of §7.4 supplies the two structural tools: the flow decomposition theorem (Lemma 7.1 — every nonzero nonnegative circulation is a positive combination f=∑iaifi\mathbf{f}=\sum_i a_i\mathbf{f}^if=∑i​ai​fi of simple circulations with only forward arcs, with integer aia_iai​ when f\mathbf{f}f is integer) and the optimality criterion for the minimum cost network flow problem (Theorem 7.6 — a feasible flow is optimal if and only if there is no unsaturated cycle with negative cost). Section 7.5 then formulates the maximum flow problem (max⁡bs\max b_smaxbs​ s.t. Af=b\mathbf{A}\mathbf{f}=\mathbf{b}Af=b, bt=−bsb_t=-b_sbt​=−bs​, bi=0b_i=0bi​=0 for i≠s,ti\ne s,ti=s,t, 0≤f≤u0\le\mathbf{f}\le\mathbf{u}0≤f≤u), defines augmenting paths (Definition 7.2: fij<uijf_{ij}<u_{ij}fij​<uij​ on forward arcs, fij>0f_{ij}>0fij​>0 on backward arcs) and the Ford–Fulkerson algorithm, and proves integer invariance and finite termination for integer capacities (Theorem 7.8). The goal is Theorem 7.10:

(a) if the Ford–Fulkerson algorithm terminates because no augmenting path can be found, the current flow is optimal;

(b) the value of the maximum flow equals the minimum cut capacity

C(S)=∑{(i,j)∈A∣i∈S, j∉S}uijC(S)=\sum_{\{(i,j)\in\mathcal{A}\mid i\in S,\,j\notin S\}}u_{ij}C(S)={(i,j)∈A∣i∈S,j∈/S}∑​uij​

— the archetypal combinatorial min-max theorem, which the book notes can also be read as LP duality (pp. 311-312).

14 thms5 active usersReviewed
🏆Completed
Bandit AlgorithmsMachine Learning·Captain: Shuze Chen

Bandit Algorithms XIII: Pure Exploration and Best-Arm IdentificationTextbook

Sometimes reward during learning is irrelevant — a pharmaceutical company running phase-II trials only cares about identifying the best treatment, as quickly and as reliably as possible. Chapter 33 of Lattimore–Szepesvári formalizes fixed-confidence best-arm identification: a policy together with a stopping time τ\tauτ and a recommendation must be sound (wrong with probability at most δ\deltaδ) while minimizing E[τ]\mathbb{E}[\tau]E[τ]. The information-theoretic complexity is c∗(ν)−1=sup⁡α∈Pk−1inf⁡ν′∈Ealt(ν)∑iαiD(νi,νi′)c^*(\nu)^{-1} = \sup_{\alpha\in\mathcal{P}_{k-1}} \inf_{\nu'\in\mathcal{E}_{alt}(\nu)} \sum_i \alpha_i D(\nu_i, \nu_i')c∗(ν)−1=supα∈Pk−1​​infν′∈Ealt​(ν)​∑i​αi​D(νi​,νi′​): every sound strategy needs E[τ]≥c∗(ν)log⁡14δ\mathbb{E}[\tau] \ge c^*(\nu)\log\frac{1}{4\delta}E[τ]≥c∗(ν)log4δ1​, and the Track-and-Stop algorithm — the goal theorem — achieves lim⁡δ→0E[τ]/log⁡(1/δ)=c∗(ν)\lim_{\delta\to 0} \mathbb{E}[\tau]/\log(1/\delta) = c^*(\nu)limδ→0​E[τ]/log(1/δ)=c∗(ν) exactly. The mission also covers the fixed-budget counterpart, sequential halving.

50 thms5 active usersReviewed
🏆Completed
Bandit AlgorithmsMachine Learning·Captain: Shuze Chen

Bandit Algorithms IV: Bernoulli Bandits and KL-UCBTextbook

When rewards are binary — a click or no click, a cure or no cure — the subgaussian machinery of Missions I–III is not tight: the variance of a Bernoulli arm degrades near the boundary of [0,1][0,1][0,1], and the correct exponential rate is governed by the binary relative entropy d(p,q)=plog⁡pq+(1−p)log⁡1−p1−qd(p,q) = p\log\frac{p}{q} + (1-p)\log\frac{1-p}{1-q}d(p,q)=plogqp​+(1−p)log1−q1−p​ rather than a squared distance. Chapter 10 of Lattimore–Szepesvári develops Chernoff's tail bound in its information-theoretic form and the KL-UCB algorithm, whose upper confidence bounds are level sets of ddd. The goal theorem shows KL-UCB attains

lim sup⁡n→∞Rn/log⁡n=∑i:Δi>0Δi/d(μi,μ∗)\limsup_{n\to\infty} R_n/\log n = \sum_{i:\Delta_i>0} \Delta_i/d(\mu_i, \mu^*)n→∞limsup​Rn​/logn=i:Δi​>0∑​Δi​/d(μi​,μ∗)

— asymptotic optimality with exactly the constant demanded by the lower bound of Mission VII, strictly improving subgaussian UCB on every Bernoulli instance.

25 thms5 active usersReviewed
🏆Completed
Bandit AlgorithmsMachine Learning·Captain: Shuze Chen

Bandit Algorithms VI: Information-Theoretic FoundationsTextbook

Every lower bound in bandit theory rests on one question: how hard is it to tell two probability measures apart from a sample? The answer is quantified by the relative entropy D(P,Q)D(P,Q)D(P,Q), and the sharpest elementary tool is the Bretagnolle–Huber inequality: for any event AAA, P(A)+Q(Ac)≥12exp⁡(−D(P,Q))P(A) + Q(A^c) \ge \frac{1}{2}\exp(-D(P,Q))P(A)+Q(Ac)≥21​exp(−D(P,Q)) — no test can distinguish PPP from QQQ with total error probability below 12e−D(P,Q)\frac{1}{2}e^{-D(P,Q)}21​e−D(P,Q). This mission formalizes Chapter 14 of Lattimore–Szepesvári: the Bretagnolle–Huber inequality (the goal theorem, proved via Le Cam's inequality ∫p∧q≥12(∫pq)2\int p \wedge q \ge \frac{1}{2}(\int\sqrt{pq})^2∫p∧q≥21​(∫pq​)2), Pinsker's inequality δ(P,Q)≤D(P,Q)/2\delta(P,Q) \le \sqrt{D(P,Q)/2}δ(P,Q)≤D(P,Q)/2​, and the closed-form divergences between Gaussians and Bernoullis. These half-page inequalities power every impossibility result in Missions VII, XI and beyond.

5 thms5 active usersReviewed
🏆Completed
OptimizationProbability·Captain: mikedeng1

Air Travel Demand and Airline Seat Inventory Management II: The EMSR Protection Level for Two Nested Fare ClassesTextbook

Why airlines protect seats

An airline sells the seats of one flight in several fare classes at different prices, all drawn from one shared cabin. Discount fares are bought early, under advance-purchase restrictions, while most high-fare requests arrive close to departure. Accepting every early low-fare request fills the aircraft with cheap passengers and turns away late high-fare passengers; refusing too many leaves seats empty. Seat inventory control decides how many seats to keep away from the low fare. Peter Belobaba's 1987 MIT thesis (Flight Transportation Laboratory Report R87-7) introduced the expected marginal seat revenue (EMSR) model for this decision, and EMSR-type rules remain the basis of the booking-limit logic in airline revenue management systems.

Timeline. Littlewood (1972, AGIFORS Symposium Proceedings; reprinted 2005) proposed accepting a low-fare request as long as its fare is at least the high fare times the probability of selling all remaining seats to high-fare passengers. Analysts at Trans World Airlines (1973) and Richter at Lufthansa (1982) gave equivalent formulations for the dynamic case. Belobaba (1987, Ch. 5) restated the two-class rule as a static protection level for nested inventories and extended it heuristically to many classes. Brumelle and McGill (Operations Research 41, 1993) and Curry (Transportation Science 24, 1990) later proved optimality of nested protection levels for any number of classes under low-to-high arrivals, and showed that Belobaba's multi-class EMSR levels are not optimal for three or more classes. This mission concerns only the two-class result, which is correct.

Setting

A single flight leg has capacity C∈NC \in \mathbb NC∈N. Class 1 has fare f1f_1f1​, class 2 has fare f2f_2f2​, with 0≤f2≤f10 \le f_2 \le f_10≤f2​≤f1​. The numbers of requests for the two classes are random variables r1,r2r_1, r_2r1​,r2​ with values in N\mathbb NN, defined on a probability space (Ω,μ)(\Omega, \mu)(Ω,μ) and independent. There are no cancellations, no no-shows, and a refused request is lost.

The inventory is nested: a class-1 request is accepted as long as any seat is unsold. A protection level S∈{0,…,C}S \in \{0, \dots, C\}S∈{0,…,C} is the number of seats reserved for class 1; it sets the class-2 booking limit BL2=C−SBL_2 = C - SBL2​=C−S. All class-2 requests arrive before any class-1 request. Class 2 therefore books min⁡(r2,C−S)\min(r_2, C - S)min(r2​,C−S) seats and class 1 books min⁡(r1,C−min⁡(r2,C−S))\min(r_1, C - \min(r_2, C-S))min(r1​,C−min(r2​,C−S)), and the realised revenue is

RS=f2min⁡(r2,C−S)+f1min⁡(r1, C−min⁡(r2,C−S)).R_S = f_2 \min(r_2, C - S) + f_1 \min\bigl(r_1,\, C - \min(r_2, C - S)\bigr).RS​=f2​min(r2​,C−S)+f1​min(r1​,C−min(r2​,C−S)).

The expected revenue is Rˉ(S)=E[RS]\bar R(S) = \mathbb E[R_S]Rˉ(S)=E[RS​].

The tail probability of class 1 is Pˉ1(S)=P[r1≥S]\bar P_1(S) = P[r_1 \ge S]Pˉ1​(S)=P[r1​≥S], the probability of receiving SSS or more class-1 requests, and the expected marginal seat revenue of the SSS-th class-1 seat is

EMSR1(S)=f1⋅Pˉ1(S).\mathrm{EMSR}_1(S) = f_1 \cdot \bar P_1(S).EMSR1​(S)=f1​⋅Pˉ1​(S).

For a single class with SSS seats the expected revenue is f1 E[min⁡(r1,S)]f_1\,\mathbb E[\min(r_1, S)]f1​E[min(r1​,S)], and EMSR1(S)\mathrm{EMSR}_1(S)EMSR1​(S) is its increment from S−1S-1S−1 to SSS seats. The EMSR protection level S21S_2^1S21​ is the largest integer S∈{0,…,C}S \in \{0, \dots, C\}S∈{0,…,C} with

EMSR1(S)≥f2.\mathrm{EMSR}_1(S) \ge f_2 .EMSR1​(S)≥f2​.

In Lean these objects are nestedRevenue, expectedNestedRevenue, tailProb, classRevenue, emsr and emsrProtectionLevel in SeatInventory.Nested.

Formalization targets

Goal: Eqs. (5.15)–(5.16), optimality of the EMSR protection level

Rˉ(S)≤Rˉ(S21)for all S∈{0,…,C}.\bar R(S) \le \bar R(S_2^1) \qquad \text{for all } S \in \{0, \dots, C\}.Rˉ(S)≤Rˉ(S21​)for all S∈{0,…,C}.

The goal fixes no distribution: it holds for every pair of independent N\mathbb NN-valued demands, and S21S_2^1S21​ depends only on f2/f1f_2/f_1f2​/f1​ and the law of r1r_1r1​.

Milestones

  1. Eq. (5.11). f1E[min⁡(r1,S)]−f1E[min⁡(r1,S−1)]=f1P[r1≥S]f_1\mathbb E[\min(r_1,S)] - f_1\mathbb E[\min(r_1,S-1)] = f_1 P[r_1 \ge S]f1​E[min(r1​,S)]−f1​E[min(r1​,S−1)]=f1​P[r1​≥S] for S≥1S \ge 1S≥1.
  2. Eqs. (6.1)–(6.2). Pˉ1\bar P_1Pˉ1​ and, for f1≥0f_1 \ge 0f1​≥0, EMSR1\mathrm{EMSR}_1EMSR1​ are non-increasing in SSS.
  3. Eq. (4.8), Littlewood's rule, already on the platform as RevenueManagement.littlewood_marginal_value (Talluri and van Ryzin's Eq. (2.1), proved).
  4. Sect. 5.2, p. 112. Rˉ(S)≤Rˉ(S21)\bar R(S) \le \bar R(S_2^1)Rˉ(S)≤Rˉ(S21​) for S21≤S≤CS_2^1 \le S \le CS21​≤S≤C: a smaller booking limit for class 2 cannot raise expected revenue.
  5. Sect. 5.2, p. 114. With the same class-2 limit C−SC - SC−S, the expected nested revenue is at least the expected revenue of two distinct inventories with SSS and C−SC - SC−S seats, strictly if f1>0f_1 > 0f1​>0 and P[r2<C−S, r1>S]>0P[r_2 < C - S,\ r_1 > S] > 0P[r2​<C−S, r1​>S]>0.

Significance

The two-class result says that, for a static booking limit set once before sales open and low-fare demand arriving first, the airline needs only the high-fare demand distribution and the fare ratio to set the optimal limit; the low-fare forecast is irrelevant. This is the rule that the thesis then applies class by class in multi-class nested systems, and it is the base case against which the later exact multi-class theory (Brumelle–McGill, Curry) is checked. Milestone 5 makes precise why nested inventories dominate the distinct-inventory allocation of the thesis's Sect. 5.1 with the same class-2 limit.

The result is classical and proved, in the sense that the optimality of a two-class threshold policy follows from Littlewood's argument and from the dynamic-programming treatment in Talluri and van Ryzin's The Theory and Practice of Revenue Management (2004, Ch. 2). On Prove2Me, Littlewood's marginal rule and the dynamic-programming optimality of nested protection levels (RevenueManagement.static_optimal_controls) are formalized, but in Bellman form: there the protection level is defined through the value function of a dynamic program. What is not formalized is the statement in Belobaba's form, where the protection level is the explicit threshold of f1P[r1≥S]f_1 P[r_1 \ge S]f1​P[r1​≥S] against f2f_2f2​ and the objective is the explicit expected revenue of a booking limit. Connecting the two forms, and the comparison with distinct inventories, is the work of this mission.

Difficulty

The expected revenue couples the two demands through the capacity left by class 2, so Rˉ\bar RRˉ is not a sum of single-class revenues and is not separately concave in an obvious way. The step that requires care is the increment Rˉ(S)−Rˉ(S−1)\bar R(S) - \bar R(S-1)Rˉ(S)−Rˉ(S−1): it is not EMSR1(S)−f2\mathrm{EMSR}_1(S) - f_2EMSR1​(S)−f2​, as the thesis's sentence after the milestone on p. 112 suggests, because the extra protected seat matters only on the event that class 2 would have reached its limit. Independence of r1r_1r1​ and r2r_2r2​ is what makes that event's probability factor out; without independence the threshold rule is not optimal. The discrete reading matters too: with P[r1>S]P[r_1 > S]P[r1​>S] in place of P[r1≥S]P[r_1 \ge S]P[r1​≥S] the rule is off by one seat and the claim fails.

Formalization scope

Conventions the Lean statements commit to:

  • Demands are N\mathbb NN-valued measurable random variables r₁ r₂ : Ω → ℕ on a probability space μ; the goal and milestone 4 assume IndepFun r₁ r₂ μ. The thesis writes continuous densities (Eqs. (5.1)–(5.5)) but requires integer seat counts; the discrete model is used throughout.
  • Pˉ1(S)=P[r1≥S]\bar P_1(S) = P[r_1 \ge S]Pˉ1​(S)=P[r1​≥S], as in Eq. (6.2) and the prose of Eq. (5.11), not P[r1>S]P[r_1 > S]P[r1​>S] as in Eq. (5.2).
  • The EMSR protection level is the largest S∈{0,…,C}S \in \{0,\dots,C\}S∈{0,…,C} with f1P[r1≥S]≥f2f_1 P[r_1 \ge S] \ge f_2f1​P[r1​≥S]≥f2​ (Eq. (5.15)); Eq. (5.16)'s equality is the continuous idealisation and is not stated.
  • Booking order: all class-2 requests precede all class-1 requests (pp. 108, 112). This order is built into the revenue formula, not assumed separately.
  • Fares satisfy 0≤f2≤f10 \le f_2 \le f_10≤f2​≤f1​; the thesis has f1>f2f_1 > f_2f1​>f2​, and the statements also cover equality.
  • Expectations are Bochner integrals of bounded revenues, probabilities are μ.real; seat counts use truncated subtraction only where S≤CS \le CS≤C.

A trivializing formalization is ruled out: S21S_2^1S21​ is defined by the threshold of (5.15), never as an argmax of expected revenue, and the expected revenue is computed from the realised revenue of the booking process, not postulated as a sum of marginal terms.

The multi-class EMSR levels of Eqs. (5.19)–(5.29) and the dynamic revision of Eqs. (5.31)–(5.32) are out of scope. Proofs need the discrete expectation identity E[min⁡(r,S)]−E[min⁡(r,S−1)]=P[r≥S]\mathbb E[\min(r,S)] - \mathbb E[\min(r,S-1)] = P[r \ge S]E[min(r,S)]−E[min(r,S−1)]=P[r≥S] and expectation of products of independent bounded functions, both in Mathlib's reach and reusable for other single-leg revenue models. Proofs of any milestone, and a proof of the goal from milestones 1, 2 and 4 plus the matching lower-half argument, are welcome.

Selected references

  • P. P. Belobaba, Air Travel Demand and Airline Seat Inventory Management, PhD thesis, MIT, Flight Transportation Laboratory Report R87-7, 1987 (no DOI).
  • K. Littlewood, Forecasting and control of passenger bookings, AGIFORS Symposium Proceedings 12, 1972; reprinted in Journal of Revenue and Pricing Management 4, 2005. https://doi.org/10.1057/palgrave.rpm.5170134
  • S. L. Brumelle and J. I. McGill, Airline seat allocation with multiple nested fare classes, Operations Research 41, 1993. https://doi.org/10.1287/opre.41.1.127
  • R. E. Curry, Optimal airline seat allocation with fare classes nested by origins and destinations, Transportation Science 24, 1990. https://doi.org/10.1287/trsc.24.3.193
  • K. T. Talluri and G. J. van Ryzin, The Theory and Practice of Revenue Management, Springer, 2004. https://doi.org/10.1007/b139000
8 thms4 active usersReviewed
🏆Completed
Linear OptimizationOptimization·Captain: mikedeng1

Linear Programming: Foundations and Extensions I: Degeneracy and Termination of the Simplex Method under Bland's RuleTextbook

Motivation

The simplex method is the standard algorithm for linear programming, and its correctness rests on one question: does it stop? Each pivot of the method moves from one dictionary to another without decreasing the objective value, but a pivot can leave the objective unchanged. When that happens repeatedly, the method can return to a dictionary it has already visited and loop forever. This behaviour, cycling, is not hypothetical: Vanderbei's Chapter 3 exhibits a problem with four decision variables and three constraints on which the "largest coefficient" entering rule with a natural tie-breaking rule cycles through six dictionaries (Vanderbei 2014, pp. 26–27).

The chapter answers the question with two pivoting rules under which the simplex method provably terminates, and then draws the consequence that makes linear programming a finite theory: the fundamental theorem of linear programming. This mission formalizes the chapter's four numbered theorems in Vanderbei's own setting of standard-form problems with slack variables.

Timeline. Hoffman (1953) and Beale (1955) gave the first examples of cycling. The perturbation and lexicographic methods go back to Charnes (1952) and to Dantzig, Orden and Wolfe (1955). Bland (1977) introduced the smallest-index rule and proved that the simplex method terminates under it (Bland 1977).

Setting

A linear program in standard form has mmm constraints and nnn decision variables:

maximize ∑j=1ncjxjsubject to∑j=1naijxj≤bi (i=1,…,m),xj≥0 (j=1,…,n).\text{maximize } \sum_{j=1}^n c_j x_j \quad\text{subject to}\quad \sum_{j=1}^n a_{ij}x_j \le b_i\ (i=1,\dots,m),\qquad x_j \ge 0\ (j=1,\dots,n).maximize j=1∑n​cj​xj​subject toj=1∑n​aij​xj​≤bi​ (i=1,…,m),xj​≥0 (j=1,…,n).

A solution xxx is feasible if it satisfies every constraint, and optimal if in addition it maximizes the objective among feasible solutions. The problem is infeasible if no feasible solution exists, and unbounded if it has feasible solutions with arbitrarily large objective values.

The slack variables wi=bi−∑jaijxjw_i = b_i - \sum_j a_{ij}x_jwi​=bi​−∑j​aij​xj​ are appended to the list of variables as xn+i=wix_{n+i} = w_ixn+i​=wi​, so that the constraints become the linear system [A I] x=b[A\ I]\,x = b[A I]x=b with x≥0x \ge 0x≥0 in Rn+m\mathbb{R}^{n+m}Rn+m. A dictionary is given by a set B\mathcal BB of mmm basic indices whose columns of [A I][A\ I][A I] are linearly independent; the remaining indices N\mathcal NN are nonbasic. Solving for the basic variables gives

ζ=ζˉ+∑j∈Ncˉjxj,xi=bˉi−∑j∈Naˉijxj(i∈B).\zeta = \bar\zeta + \sum_{j\in\mathcal N}\bar c_j x_j,\qquad x_i = \bar b_i - \sum_{j\in\mathcal N}\bar a_{ij}x_j\quad (i\in\mathcal B).ζ=ζˉ​+j∈N∑​cˉj​xj​,xi​=bˉi​−j∈N∑​aˉij​xj​(i∈B).

The basic solution of the dictionary sets the nonbasic variables to zero. The dictionary is feasible if bˉi≥0\bar b_i \ge 0bˉi​≥0 for every i∈Bi\in\mathcal Bi∈B, and degenerate if bˉi=0\bar b_i = 0bˉi​=0 for some i∈Bi\in\mathcal Bi∈B.

The simplex method (Phase II) starts at a feasible dictionary and repeats a pivot: an entering variable xkx_kxk​ is chosen among the nonbasic variables with cˉk>0\bar c_k > 0cˉk​>0, and a leaving variable xlx_lxl​ among the basic variables with aˉlk>0\bar a_{lk} > 0aˉlk​>0 that minimize the ratio bˉl/aˉlk\bar b_l/\bar a_{lk}bˉl​/aˉlk​; then xkx_kxk​ becomes basic and xlx_lxl​ nonbasic. The method stops when no cˉj\bar c_jcˉj​ is positive (the dictionary is optimal) or when the entering column has no positive aˉik\bar a_{ik}aˉik​ (the problem is unbounded). A pivoting rule resolves the remaining choices. Bland's rule chooses both the entering and the leaving variable as the candidate with the smallest index. The lexicographic rule perturbs the right-hand sides by symbols 0<ϵm≪⋯≪ϵ1≪0<\epsilon_m\ll\dots\ll\epsilon_1\ll0<ϵm​≪⋯≪ϵ1​≪ all data and chooses the leaving variable by the perturbed ratio test.

Formalization targets

Goal: Theorem 3.3 (termination under Bland's rule, p. 31)

From every feasible dictionary D0D_0D0​, there is no infinite sequence of pivots

D0→D1→D2→⋯D_0\to D_1\to D_2\to\cdotsD0​→D1​→D2​→⋯

in which both the entering and the leaving variable follow Bland's rule; and a finite sequence of such pivots reaches a dictionary DTD_TDT​ at which the method stops, optimal or exhibiting unboundedness.

Milestones

  • Theorem 3.1 (p. 27): if the simplex method fails to terminate, it must cycle, i.e. an infinite run visits some dictionary twice.
  • Theorem 3.2 (p. 30): the simplex method always terminates when the leaving variable is selected by the lexicographic rule.
  • Theorem 3.4 (p. 33), the fundamental theorem: (1) with no optimal solution the problem is infeasible or unbounded; (2) if a feasible solution exists, a basic feasible solution exists; (3) if an optimal solution exists, a basic optimal solution exists.

Significance

The result. Termination under Bland's rule is what turns the simplex method from a heuristic into an algorithm. Combined with Phase I, it yields the fundamental theorem of linear programming, which reduces the search for an optimum to finitely many basic solutions and underlies the duality theory of the following chapters. Bland's rule also needs no perturbation or extra bookkeeping, and it is the anticycling rule used in many correctness proofs of simplex-type and combinatorial pivoting algorithms, including oriented-matroid programming.

Formalizing it. All four theorems are classical and proved. The Prove2Me library has the lexicographic rule and a nondegenerate termination theorem in the tableau setting of Bertsimas and Tsitsiklis (equality form Ax=bAx=bAx=b, x≥0x\ge 0x≥0, full row rank), and the existence of basic feasible and optimal solutions in that form. It has no statement of Bland's theorem, and none of Vanderbei's dictionary formulation over [A I][A\ I][A I]. This mission produces a machine-checkable model of dictionaries and pivoting rules in that formulation, and targets Bland's theorem, whose proof is a genuine combinatorial argument rather than a monotonicity argument.

Difficulty

The natural argument for termination is monotonicity: each pivot increases the objective, so no dictionary repeats. It fails exactly at degenerate pivots, where the step length bˉl/aˉlk\bar b_l/\bar a_{lk}bˉl​/aˉlk​ is zero and the objective and the basic solution do not change. Bland's rule gives no potential function that strictly increases along degenerate pivots, so the proof has to reason about a hypothetical cycle as a whole: which variables enter and leave the basis within it, and how two dictionaries of the cycle, in which the same variable leaves and later enters, constrain each other's coefficients. Relating the coefficients of two different dictionaries of the same problem is the step that has no counterpart in the model's definitions and has to be developed.

For Theorem 3.2, the symbols ϵi\epsilon_iϵi​ cannot be replaced by a fixed small real number: the method treats them as formal quantities on separate scales, and the statement is about that symbolic rule.

Formalization scope

Vectors are Fin n → ℝ, Fin m → ℝ, and the constraint matrix is Matrix (Fin m) (Fin n) ℝ. The n+mn+mn+m variables are indexed by Fin (n + m) with the decision variables first and the slacks after them, which is the order x1,…,xn,xn+1=w1,…,xn+m=wmx_1,\dots,x_n,x_{n+1}=w_1,\dots,x_{n+m}=w_mx1​,…,xn​,xn+1​=w1​,…,xn+m​=wm​ that Bland's rule compares. A dictionary is a structure holding its basic set, a proof that it has mmm elements and a proof that its columns of [A I][A\ I][A I] are linearly independent; the coefficients bˉ,aˉ,cˉ,ζˉ\bar b,\bar a,\bar c,\bar\zetabˉ,aˉ,cˉ,ζˉ​ are computed as coordinates in the basis of basic columns. A dictionary is therefore determined by its basic set, as the proof of Theorem 3.1 uses. "Basic solution" is defined through such a dictionary, not as a support condition.

Termination is stated as the nonexistence of an infinite run from a feasible dictionary, for Theorems 3.2 and 3.3. The goal adds that a finite Bland run reaches a stopping dictionary, so that it cannot hold because pivots fail to exist. The lexicographic rule is encoded by lexicographic comparison of the coefficient vectors (bˉi,ri1,…,rim)/aˉik(\bar b_i, r_{i1},\dots,r_{im})/\bar a_{ik}(bˉi​,ri1​,…,rim​)/aˉik​ of the perturbed ratios; the symbol ϵp\epsilon_pϵp​ is attached in the starting dictionary to its ppp-th basic variable in increasing index order, which for the initial dictionary is the ppp-th constraint. Unboundedness in Theorem 3.4 is "for every MMM a feasible solution with objective >M>M>M", as defined on p. 7. The statements carry no explicit constants.

A formalization that stated Theorem 3.3 for arbitrary pivot sequences with pairwise distinct bases would be Theorem 3.1's counting argument, not Bland's theorem; the goal is stated for pivots that follow Bland's rule and only those.

A complete development needs: the pivot update of a dictionary and the invariance of the solution set under it, feasibility preservation by the ratio test, the relation between the objective rows of two dictionaries, and finiteness of the set of bases. These are reusable for any later formalization of simplex-type algorithms. Proofs of the milestones, alternative proofs of Theorem 3.3, and Phase I (to connect Theorem 3.4 with the algorithm) are welcome.

Selected references

  • R. J. Vanderbei, Linear Programming: Foundations and Extensions, 4th ed., International Series in Operations Research & Management Science 196, Springer, 2014, Chapter 3. https://doi.org/10.1007/978-1-4614-7630-6
  • R. G. Bland, New finite pivoting rules for the simplex method, Mathematics of Operations Research 2(2):103–107, 1977. https://doi.org/10.1287/moor.2.2.103
  • G. B. Dantzig, A. Orden, P. Wolfe, The generalized simplex method for minimizing a linear form under linear inequality restraints, Pacific Journal of Mathematics 5(2):183–195, 1955. https://doi.org/10.2140/pjm.1955.5.183
  • E. M. L. Beale, Cycling in the dual simplex algorithm, Naval Research Logistics Quarterly 2(4):269–275, 1955. https://doi.org/10.1002/nav.3800020406
  • D. Bertsimas, J. N. Tsitsiklis, Introduction to Linear Optimization, Athena Scientific, 1997, §3.4 (lexicographic rule and Bland's rule in tableau form).
10 thms4 active usersReviewed
🏆Completed
Dynamic ProgrammingProbabilityStochastic Systems·Captain: mikedeng1

Stochastic Dynamic Programming and the Control of Queueing Systems III: Approximating Sequences for the Discounted Cost CriterionTextbook

Motivation

Optimal control of queueing systems leads to Markov decision problems whose state space is countably infinite (buffer contents, numbers of customers) and whose costs are unbounded (holding costs grow with the queue). Such a problem cannot be solved on a computer as it stands. The standard remedy is to truncate: solve a finite problem on the states {0,1,…,N}\{0,1,\dots,N\}{0,1,…,N} and hope that its value and its optimal policy approximate those of the original problem as NNN grows. Linn Sennott's approximating sequence method (ASM) makes this hope precise. For the expected discounted cost criterion, Sections 4.6–4.7 of Sennott's book (Stochastic Dynamic Programming and the Control of Queueing Systems, Wiley, 1999) identify a single condition, Assumption DC(α\alphaα), that is necessary and sufficient for convergence of the truncated values, and give checkable sufficient conditions for it.

The method matters because naive truncation can fail. The book's Example 4.6.1 has a chain whose value at state 000 is finite, yet a natural truncation produces values VαN(0)≥αN2/((1−α)[N(1−α)+α])→∞V^N_\alpha(0)\ge \alpha N^2/((1-\alpha)[N(1-\alpha)+\alpha])\to\inftyVαN​(0)≥αN2/((1−α)[N(1−α)+α])→∞. How the probability that would leave the truncated set is redistributed decides whether the computation is meaningful.

Earlier truncation schemes (Fox 1971; White 1980, 1982; Hernández-Lerma 1986; Cavazos-Cadena 1986; Whitt 1978–79; see the bibliographic notes on p. 81 of the book and Puterman 1994) require bounded rewards or pass directly to an algorithm. The ASM instead produces a sequence of finite Markov decision chains that can be studied in their own right; the material of Sections 4.6–4.7 is presented in the book as new.

Setting

A Markov decision chain (MDC) Δ\DeltaΔ has a countable state space SSS; for each state iii a finite nonempty action set AiA_iAi​; nonnegative finite costs C(i,a)C(i,a)C(i,a); and transition probabilities Pij(a)P_{ij}(a)Pij​(a) with ∑jPij(a)=1\sum_j P_{ij}(a)=1∑j​Pij​(a)=1. A policy θ\thetaθ chooses the action at time ttt at random from a distribution θ(⋅∣ht)\theta(\cdot\mid h_t)θ(⋅∣ht​) on AitA_{i_t}Ait​​ that may depend on the entire history ht=(i0,a0,…,it−1,at−1,it)h_t=(i_0,a_0,\dots,i_{t-1},a_{t-1},i_t)ht​=(i0​,a0​,…,it−1​,at−1​,it​). A stationary policy fff always chooses f(i)∈Aif(i)\in A_if(i)∈Ai​ in state iii. Fix a discount factor α∈(0,1)\alpha\in(0,1)α∈(0,1). The discounted cost of θ\thetaθ and the discounted value function are

Vθ,α(i)=∑t≥0αtEθ[C(Xt,At)∣X0=i],Vα(i)=inf⁡θVθ,α(i),V_{\theta,\alpha}(i)=\sum_{t\ge0}\alpha^tE_\theta[C(X_t,A_t)\mid X_0=i],\qquad V_\alpha(i)=\inf_\theta V_{\theta,\alpha}(i),Vθ,α​(i)=t≥0∑​αtEθ​[C(Xt​,At​)∣X0​=i],Vα​(i)=θinf​Vθ,α​(i),

both in [0,∞][0,\infty][0,∞], the infimum over all policies. A policy is discount optimal if Vθ,α=VαV_{\theta,\alpha}=V_\alphaVθ,α​=Vα​.

An approximating sequence (ΔN)N≥N0(\Delta_N)_{N\ge N_0}(ΔN​)N≥N0​​ consists of finite nonempty sets SNS_NSN​ increasing to SSS and, for i∈SNi\in S_Ni∈SN​ and a∈Aia\in A_ia∈Ai​, probability distributions Pij(a;N)P_{ij}(a;N)Pij​(a;N) on SNS_NSN​ with Pij(a;N)→Pij(a)P_{ij}(a;N)\to P_{ij}(a)Pij​(a;N)→Pij​(a) as N→∞N\to\inftyN→∞. The finite MDC ΔN\Delta_NΔN​ has state space SNS_NSN​ and the same actions and costs; VαNV^N_\alphaVαN​ is its value function and fαNf^N_\alphafαN​ a stationary policy attaining the minimum in its discount optimality equation

VαN(i)=min⁡a∈Ai{C(i,a)+α∑j∈SNPij(a;N)VαN(j)},i∈SN.V^N_\alpha(i)=\min_{a\in A_i}\Big\{C(i,a)+\alpha\sum_{j\in S_N}P_{ij}(a;N)V^N_\alpha(j)\Big\},\qquad i\in S_N.VαN​(i)=a∈Ai​min​{C(i,a)+αj∈SN​∑​Pij​(a;N)VαN​(j)},i∈SN​.

An augmentation type approximating sequence (ATAS) keeps the original probabilities inside SNS_NSN​ and redistributes the excess probability Pir(a)P_{ir}(a)Pir​(a), r∉SNr\notin S_Nr∈/SN​, according to augmentation distributions qj(i,a,r,N)q_j(i,a,r,N)qj​(i,a,r,N) on SNS_NSN​: Pij(a;N)=Pij(a)+∑r∉SNPir(a)qj(i,a,r,N)P_{ij}(a;N)=P_{ij}(a)+\sum_{r\notin S_N}P_{ir}(a)q_j(i,a,r,N)Pij​(a;N)=Pij​(a)+∑r∈/SN​​Pir​(a)qj​(i,a,r,N).

Assumption DC(α\alphaα): for every i∈Si\in Si∈S, Wα(i):=lim sup⁡NVαN(i)<∞W_\alpha(i):=\limsup_{N}V^N_\alpha(i)<\inftyWα​(i):=limsupN​VαN​(i)<∞ and Wα(i)≤Vα(i)W_\alpha(i)\le V_\alpha(i)Wα​(i)≤Vα​(i).

Formalization targets

Goal: Theorem 4.6.3

The following are equivalent:

(i) lim⁡N→∞VαN(i)=Vα(i)<∞  (i∈S);(ii) Assumption DC(α).\text{(i)}\ \lim_{N\to\infty}V^N_\alpha(i)=V_\alpha(i)<\infty\ \ (i\in S);\qquad \text{(ii)}\ \text{Assumption DC}(\alpha).(i) N→∞lim​VαN​(i)=Vα​(i)<∞  (i∈S);(ii) Assumption DC(α).

Under either, every limit point of (fαN)N≥N0(f^N_\alpha)_{N\ge N_0}(fαN​)N≥N0​​ (a stationary fff with fNr(i)=f(i)f^{N_r}(i)=f(i)fNr​(i)=f(i) eventually along a subsequence, for each iii) is discount optimal for Δ\DeltaΔ.

Milestones

  • Lemma 4.6.2: lim inf⁡NVαN≥Vα\liminf_N V^N_\alpha\ge V_\alphaliminfN​VαN​≥Vα​ for every approximating sequence.
  • Proposition 4.7.1: bounded costs imply DC(α\alphaα).
  • Lemma 4.7.2: taboo probabilities of avoiding S−SNS-S_NS−SN​ converge to the ttt-step transition probabilities.
  • Lemma 4.7.3: for the first passage time Ti(N)T_i(N)Ti​(N) out of SNS_NSN​ under a stationary policy, E[αTi(N)]→0E[\alpha^{T_i(N)}]\to0E[αTi​(N)]→0.
  • Proposition 4.7.4: if Vα<∞V_\alpha<\inftyVα​<∞ and the ATAS sends excess probability to a finite set, DC(α\alphaα) holds.
  • Corollary 4.7.5: the case of a single distinguished state zzz, with the relative form of the optimality equation for ΔN\Delta_NΔN​.
  • Proposition 4.7.6: if Vα<∞V_\alpha<\inftyVα​<∞ and the augmentation distributions satisfy ∑j∈SNqj(i,a,r,N)vα,n(j)≤vα,n(r)\sum_{j\in S_N}q_j(i,a,r,N)v_{\alpha,n}(j)\le v_{\alpha,n}(r)∑j∈SN​​qj​(i,a,r,N)vα,n​(j)≤vα,n​(r) for all n≥0n\ge0n≥0, then VαN≤VαV^N_\alpha\le V_\alphaVαN​≤Vα​ on SNS_NSN​.

Significance

Theorem 4.6.3 turns the question "does truncation work?" into the verification of one inequality between a lim sup and the true value, and it delivers both the value and an optimal stationary policy from finite computations. Propositions 4.7.4–4.7.6 give conditions that hold in the queueing models of the book with unbounded holding costs, and Corollary 4.7.5 supplies the computational form used for the inventory model of Chapter 5. The discounted theory is also the stepping stone to the average cost ASM of Chapter 8, which is built on discounted approximations.

All results are proved in the book. None of them is formalized: the platform has no statement about approximating sequences or state truncation of countable-state MDPs, and Mathlib has no Markov decision processes. The mission produces machine-checked versions of the convergence theorem and its sufficient conditions, for general history-dependent randomized policies and [0,∞][0,\infty][0,∞]-valued costs.

Difficulty

The value functions are infima over uncountably many history-dependent policies and may be infinite, so no contraction argument applies: costs are unbounded and VαV_\alphaVα​ is only the minimal nonnegative solution of its optimality equation. Passing to the limit in NNN inside ∑j∈SNPij(a;N)VαN(j)\sum_{j\in S_N}P_{ij}(a;N)V^N_\alpha(j)∑j∈SN​​Pij​(a;N)VαN​(j) is an interchange of limit and infinite sum under a moving probability measure, with no dominating function in general; Example 4.6.1 shows that the interchange genuinely fails. The upper bound of Proposition 4.7.4 requires comparing ΔN\Delta_NΔN​ with Δ\DeltaΔ along a coupled first passage out of SNS_NSN​, which needs the taboo-probability estimates of Lemmas 4.7.2–4.7.3. The obvious idea of bounding VαNV^N_\alphaVαN​ by sup⁡C/(1−α)\sup C/(1-\alpha)supC/(1−α) works only for bounded costs (Proposition 4.7.1).

Formalization scope

The state type S is countable ([Countable S]); actions live in a type Act, with a finite nonempty Finset of admissible actions per state. Costs are ℝ≥0, transition probabilities and all value functions are ℝ≥0∞, so infima over policies are lattice infima and +∞+\infty+∞ is a legitimate value. A general policy is a function of the history, encoded as the list of past state–action pairs (most recent first) and the current state; the expected cost at time ttt is the [0,∞][0,\infty][0,∞]-valued sum over histories. VαV_\alphaVα​ is the infimum over all such policies; a stationary policy enters as the policy putting mass one on f(i)f(i)f(i). The discount factor is α : ℝ≥0 with 0<α<10<\alpha<10<α<1 (the chapter's standing assumption). ΔN\Delta_NΔN​ is an MDC on the subtype SNS_NSN​; VαN(i)V^N_\alpha(i)VαN​(i) is extended by 000 when N<N0N<N_0N<N0​ or i∉SNi\notin S_Ni∈/SN​, a convention that affects finitely many NNN for each fixed iii and hence no limit in NNN. Limits, lim sups and lim infs are along Filter.atTop in ℝ≥0∞. Taboo probabilities and the first passage quantity E[αT]=∑n≥1αnP(T=n)E[\alpha^{T}]=\sum_{n\ge1}\alpha^nP(T=n)E[αT]=∑n≥1​αnP(T=n) (so α∞=0\alpha^\infty=0α∞=0) are defined combinatorially from the transition probabilities.

A trivializing formalization is ruled out: VαV_\alphaVα​ is not an infimum over stationary policies only (which would make optimality of limit points close to definitional), DC(α\alphaα) keeps both of its conditions, and statement (i) of the goal includes finiteness of VαV_\alphaVα​.

A complete development needs the minimality of VαV_\alphaVα​ among nonnegative solutions of the discount optimality equation (Theorem 4.1.4, chunk II of this series), Fatou-type lemmas for sums against converging distributions (Appendix A, chunk XI), and compactness of stationary policies (Proposition B.5). Contributions of these as reusable lemmas about countable-state MDCs are welcome.

Selected references

  • L. I. Sennott, Stochastic Dynamic Programming and the Control of Queueing Systems, Wiley, 1999, Sections 4.6–4.7, pp. 73–81. https://doi.org/10.1002/9780470317037
  • M. L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming, Wiley, 1994. https://doi.org/10.1002/9780470316887
11 thms4 active usersReviewed
🏆Completed
Algorithmic Game TheoryCombinatorics·Captain: mikedeng1

Theory of Games and Economic Behavior IX: Solutions for Acyclic RelationsTextbook

Motivation

The solution concept of von Neumann and Morgenstern's Theory of Games and Economic Behavior (1944) is defined from two ingredients: a set of imputations and a domination relation between them. A solution is a set of imputations that is internally stable (no member dominates another) and externally stable (every non-member is dominated by some member). In §65 of the book the authors observe that this definition never uses what imputations and domination actually are. They abstract it to an arbitrary set DDD and an arbitrary relation S\mathcal SS on DDD, and ask which properties of S\mathcal SS guarantee that exactly one solution exists.

The abstract notion is what graph theory now calls a kernel of a directed graph: draw an arc x→yx \to yx→y whenever xSyx\mathcal S yxSy; a solution is a set of vertices that is independent and absorbs every vertex outside it. Kernels appear in combinatorial game theory (the losing positions of a finite impartial game form a kernel of its move graph) and in the theory of preference and choice.

Timeline.

  • 1944 (1st ed.; 3rd ed. 1953, reprinted 2007): von Neumann and Morgenstern define solutions for an arbitrary relation (§65), show that a finite set with an acyclic relation has exactly one solution (65:X), and that acyclicity is necessary for every subset to have a unique solution (65:Z).
  • 1953: M. Richardson, Solutions of irreflexive relations, extends existence (not uniqueness) to finite relations without cycles of odd length.

Setting

Let DDD be an arbitrary set and S\mathcal SS an arbitrary relation on DDD; xSyx\mathcal S yxSy is read "xxx dominates yyy". A solution (in DDD for S\mathcal SS) is a set V⊆DV \subseteq DV⊆D with

(65:1)V={ y∈D:xSy holds for no x∈V }.\text{(65:1)}\qquad V = \{\, y \in D : x\mathcal S y \text{ holds for no } x \in V \,\}.(65:1)V={y∈D:xSy holds for no x∈V}.

For E⊆DE \subseteq DE⊆D, an element xxx is a maximum of EEE if x∈Ex \in Ex∈E and no y∈Ey \in Ey∈E has ySxy\mathcal S xySx; the set of maxima is EmE^mEm.

For m≥1m \ge 1m≥1, condition (Am)(A_m)(Am​) says: never x1Sx0,x2Sx1,…,xmSxm−1x_1\mathcal S x_0, x_2\mathcal S x_1, \dots, x_m\mathcal S x_{m-1}x1​Sx0​,x2​Sx1​,…,xm​Sxm−1​ with x0=xmx_0 = x_mx0​=xm​ and all xi∈Dx_i \in Dxi​∈D. The relation is acyclic if it satisfies every (Am)(A_m)(Am​), m=1,2,…m = 1, 2, \dotsm=1,2,…; in particular never xSxx\mathcal S xxSx. It is strictly acyclic if there is no infinite sequence x0,x1,x2,…x_0, x_1, x_2, \dotsx0​,x1​,x2​,… in DDD with xi+1Sxix_{i+1}\mathcal S x_ixi+1​Sxi​ for every iii. Property (65:K) says that every non-empty E⊆DE \subseteq DE⊆D has Em≠⊖E^m \ne \ominusEm=⊖. A partial ordering (65:B) is a transitive relation for which at most one of x=yx = yx=y, xSyx\mathcal S yxSy, ySxy\mathcal S xySx holds.

For the main theorem the book constructs a candidate solution by induction (65.7.1): A1=DA_1 = DA1​=D; Bi=AimB_i = A_i^mBi​=Aim​; CiC_iCi​ is the set of elements of AiA_iAi​ dominated by some element of BiB_iBi​; Ai+1=Ai−Bi−CiA_{i+1} = A_i - B_i - C_iAi+1​=Ai​−Bi​−Ci​. With i0i_0i0​ the first index for which Ai0=⊖A_{i_0} = \ominusAi0​​=⊖,

(65:2)V0=B1∪⋯∪Bi0−1.\text{(65:2)}\qquad V_0 = B_1 \cup \cdots \cup B_{i_0 - 1}.(65:2)V0​=B1​∪⋯∪Bi0​−1​.

In Lean the elements live in a type α, D V : Set α, and S : α → α → Prop with S x y meaning xSyx\mathcal S yxSy; the predicates are IsSolution D S V, maxima E S, IsAcyclic, IsStrictlyAcyclic, HasMaximaProperty, IsPartialOrdering, ConditionG, and the construction stageA, stageB, stageC, V0.

Formalization targets

Goal: (65:X)

If DDD is finite and S\mathcal SS is acyclic on DDD, then

∃! V: V is a solution in D for S,andV is a solution  ⟺  V=V0.\exists!\, V:\ V \text{ is a solution in } D \text{ for } \mathcal S, \qquad\text{and}\qquad V \text{ is a solution} \iff V = V_0 .∃!V: V is a solution in D for S,andV is a solution⟺V=V0​.

Milestones, in attack order

  1. (65:I) For a partial ordering, a finite DDD satisfies (65:G): every non-maximal yyy is dominated by some maximum.
  2. (65:H) For a partial ordering of an arbitrary DDD: VVV is a solution   ⟺  \iff⟺ (65:G) holds and V=DmV = D^mV=Dm.
  3. (65:O:c) Strict acyclicity implies acyclicity; for finite DDD the two are equivalent.
  4. (65:P) (65:K)   ⟺  \iff⟺ strict acyclicity, for arbitrary DDD.
  5. (65:S) For finite DDD and acyclic S\mathcal SS, some AiA_iAi​ is empty.
  6. (65:V) For finite DDD and acyclic S\mathcal SS, every solution equals V0V_0V0​.
  7. (65:W) For finite DDD and acyclic S\mathcal SS, V0V_0V0​ is a solution.
  8. (65:Z) If every E⊆DE \subseteq DE⊆D has a unique solution in EEE for S\mathcal SS, then S\mathcal SS is acyclic on DDD.

Significance

The result itself. (65:X) is the most general of the book's three existence-and-uniqueness theorems for solutions (complete ordering, partial ordering, acyclic relation; 65.8.1). For games proper it has no direct application: the set of imputations of an essential game has no maxima, so (65:K) fails (65.9.1). Its role is to isolate a sufficient condition for a unique solution. With (65:Z), and applied to every subset of DDD, it characterizes the finite relations for which every subset has exactly one solution: exactly the acyclic ones (65.8.2). In graph language it is the statement that a finite directed acyclic graph has exactly one kernel. In combinatorial game theory this is the partition of the positions of a finite impartial game into P- and N-positions. The complete- and partial-ordering results (65:E)–(65:I) are the special cases the book treats first.

Formalizing it. The results are classical and fully proved in the book; to the best of our knowledge none of them is on the Prove2Me platform, and Mathlib has well-foundedness (WellFounded, RelEmbedding of ℕ) but no kernel or von Neumann–Morgenstern solution notion for an abstract relation. The mission produces machine-checked proofs of the book's §65 chain: the equivalence of (65:K) with strict acyclicity for arbitrary sets, the finite equivalence of acyclicity and strict acyclicity, the explicit construction of V0V_0V0​, and the characterization of 65.8.2.

Difficulty

Most of the individual steps are short. The work is in making the book's finite induction precise. The sets AiA_iAi​ are defined recursively and V0V_0V0​ refers to the first empty stage i0i_0i0​. The uniqueness proof (65:V) is a minimal-counterexample argument over the stage index, which moves between "smallest kkk with y∉Aky \notin A_ky∈/Ak​" and the disjoint decomposition (65:U) of DDD into the BiB_iBi​ and CiC_iCi​. A tempting shortcut, taking an arbitrary well-founded rank function instead of the book's construction, proves existence and uniqueness but not that the solution is the V0V_0V0​ of (65:2), which is part of the goal. For (65:P) and (65:O:c) the difficulty is the passage between finite cycles and infinite chains. Going from a chain in a finite set to a repetition needs a pigeonhole argument, and going from a set without maxima to a chain needs dependent choice.

Formalization scope

  • Representation. An ambient type α; D, E, V are Set α; the relation is S : α → α → Prop and is only ever consulted on elements of the set under consideration, so it is the book's relation on DDD (or its restriction to EEE). Finite and infinite sequences are functions ℕ → α.
  • Solutions. IsSolution D S V is the set equation (65:1) literally; it forces V⊆DV \subseteq DV⊆D. Uniqueness in the goal is ∃! over all V : Set α, not over a subtype; there is no degenerate reading in which the solution is fixed by construction.
  • Acyclicity. IsAcyclic D S requires (Am)(A_m)(Am​) for every m≥1m \ge 1m≥1, all cycle elements in DDD. The case m=0m = 0m=0 is excluded, as in the book (it would be unsatisfiable). This is equivalent to the absence of a Relation.TransGen loop inside DDD, but the book's form is stated.
  • Construction. Stages are indexed from 000: stageA D S k is the book's Ak+1A_{k+1}Ak+1​. V0 D S is the union of all BiB_iBi​, which equals B1∪⋯∪Bi0−1B_1 \cup \cdots \cup B_{i_0 - 1}B1​∪⋯∪Bi0​−1​ because every later BiB_iBi​ is empty.
  • Standing hypotheses instantiated. (65:S), (65:V), (65:W) and the goal (65:X) carry the hypotheses of 65.7.1, "DDD finite and S\mathcal SS acyclic" (for finite DDD equivalently strictly acyclic, i.e. (65:K)), as D.Finite and IsAcyclic D S. (65:H) and (65:I) carry the partial-ordering hypothesis (65:B:a), (65:B:b) of 65.5.1, and (65:I) also finiteness of DDD. (65:O:c), (65:P) and (65:Z) are for arbitrary DDD and S\mathcal SS, as 65.6.2 and 65.8.2 state. The empty DDD is allowed everywhere; there the unique solution is ⊖\ominus⊖.
  • Not stated. The infinite case of (65:X) and of (65:Y), which the book leaves open (65.7.1, 65.8.3, question (65:9)); the complete-ordering results (65:E), (65:F), which silently assume D≠⊖D \neq \ominusD=⊖; the counting statement (65:8).
  • Needed infrastructure. Finite-set induction and pigeonhole on Set.Finite, dependent choice for (65:P). The definitions are reusable for any later work on kernels of digraphs and on abstract stable sets. Proofs of any milestone, and alternative proofs of the goal, are welcome.

Selected references

  • J. von Neumann and O. Morgenstern, Theory of Games and Economic Behavior, 60th-anniversary edition, Princeton University Press, 2007 (reprint of the 3rd ed., 1953), §65, pp. 587–602. https://doi.org/10.1515/9781400829460
  • M. Richardson, Solutions of irreflexive relations, Annals of Mathematics 58 (1953), 573–590. https://doi.org/10.2307/1969755
13 thms4 active usersReviewed
🏆Completed
Algorithmic Game TheoryConvex OptimizationLinear Optimization·Captain: mikedeng1

Theory of Games and Economic Behavior III: Mixed Strategies, the Minimax Theorem and Good StrategiesTextbook

Motivation

A zero-sum two-person game in normalized form is a real matrix H(τ1,τ2)\mathcal H(\tau_1, \tau_2)H(τ1​,τ2​): player 1 chooses a row τ1\tau_1τ1​, player 2 simultaneously chooses a column τ2\tau_2τ2​, and player 2 pays player 1 the amount H(τ1,τ2)\mathcal H(\tau_1, \tau_2)H(τ1​,τ2​). Matrix games are the base case of non-cooperative game theory, the prototype of every minimax statement in optimization, statistics (Wald's decision theory) and online learning, and, through their equivalence with linear programming, a standard tool of operations research.

Chapter III of von Neumann and Morgenstern's Theory of Games and Economic Behavior (1944; third edition 1953) gives the book's complete solution of these games. Timeline:

  • 1928. J. von Neumann, "Zur Theorie der Gesellschaftsspiele", Math. Annalen 100, proves that every matrix game has a value in mixed strategies (the minimax theorem), by a topological argument. https://doi.org/10.1007/BF01448847
  • 1937. von Neumann's growth-model paper gives a second proof via a fixed-point argument, later generalized by Kakutani (1941).
  • 1938. J. Ville gives the first elementary proof, based on convexity.
  • 1944. The Theory of Games presents Ville's route: a theorem of the alternative for matrices (§16) yields the minimax theorem (17:6), from which §17 derives the structure of the sets of good strategies.
  • 1951. Gale, Kuhn and Tucker, and Dantzig, relate matrix games to linear-programming duality.

Setting

Player 1 has β1≥1\beta_1 \ge 1β1​≥1 pure strategies τ1\tau_1τ1​, player 2 has β2≥1\beta_2 \ge 1β2​≥1 pure strategies τ2\tau_2τ2​, and H\mathcal HH is an arbitrary real β1×β2\beta_1 \times \beta_2β1​×β2​ matrix (14.1.1). A mixed strategy of player 1 is a probability vector ξ\xiξ in the simplex

Sβ1={ξ∈Rβ1:ξτ1≥0, ∑τ1ξτ1=1},S_{\beta_1} = \Big\{ \xi \in \mathbb R^{\beta_1} : \xi_{\tau_1} \ge 0,\ \sum_{\tau_1} \xi_{\tau_1} = 1 \Big\},Sβ1​​={ξ∈Rβ1​:ξτ1​​≥0, τ1​∑​ξτ1​​=1},

and similarly η∈Sβ2\eta \in S_{\beta_2}η∈Sβ2​​ for player 2. The pure strategy τ\tauτ is the coordinate vector δτ\delta^{\tau}δτ. The expected payoff is the bilinear form (17:2)

K(ξ,η)=∑τ1=1β1∑τ2=1β2H(τ1,τ2) ξτ1ητ2.K(\xi, \eta) = \sum_{\tau_1=1}^{\beta_1} \sum_{\tau_2=1}^{\beta_2} \mathcal H(\tau_1, \tau_2)\, \xi_{\tau_1} \eta_{\tau_2}.K(ξ,η)=τ1​=1∑β1​​τ2​=1∑β2​​H(τ1​,τ2​)ξτ1​​ητ2​​.

The good strategies of player 1 form the set Aˉ\bar AAˉ of those ξ∈Sβ1\xi \in S_{\beta_1}ξ∈Sβ1​​ at which Min⁡ηK(ξ,η)\operatorname{Min}_\eta K(\xi, \eta)Minη​K(ξ,η) assumes its maximum; those of player 2 form the set Bˉ\bar BBˉ of those η∈Sβ2\eta \in S_{\beta_2}η∈Sβ2​​ at which Max⁡ξK(ξ,η)\operatorname{Max}_\xi K(\xi, \eta)Maxξ​K(ξ,η) assumes its minimum ((17:B:a), (17:B:b)). A saddle point of KKK is a pair with K(ξ′,η)≤K(ξ,η)≤K(ξ,η′)K(\xi', \eta) \le K(\xi, \eta) \le K(\xi, \eta')K(ξ′,η)≤K(ξ,η)≤K(ξ,η′) for all ξ′,η′\xi', \eta'ξ′,η′. With pure strategies alone one has v1=Max⁡τ1Min⁡τ2Hv_1 = \operatorname{Max}_{\tau_1}\operatorname{Min}_{\tau_2}\mathcal Hv1​=Maxτ1​​Minτ2​​H and v2=Min⁡τ2Max⁡τ1Hv_2 = \operatorname{Min}_{\tau_2}\operatorname{Max}_{\tau_1}\mathcal Hv2​=Minτ2​​Maxτ1​​H; the game is specially strictly determined when v1=v2v_1 = v_2v1​=v2​.

For a general real function ϕ(x,y)\phi(x, y)ϕ(x,y) (§13) the same notions are Max⁡xMin⁡yϕ\operatorname{Max}_x \operatorname{Min}_y \phiMaxx​Miny​ϕ, Min⁡yMax⁡xϕ\operatorname{Min}_y \operatorname{Max}_x \phiMiny​Maxx​ϕ, saddle points, and the sets AϕA^\phiAϕ (maximizers of Min⁡yϕ\operatorname{Min}_y \phiMiny​ϕ) and BϕB^\phiBϕ (minimizers of Max⁡xϕ\operatorname{Max}_x \phiMaxx​ϕ), always under the book's standing hypothesis that these maxima and minima exist.

Formalization targets

Goal: (17:D), good strategies characterized by their supports

For all ξ∈Sβ1\xi \in S_{\beta_1}ξ∈Sβ1​​ and η∈Sβ2\eta \in S_{\beta_2}η∈Sβ2​​: ξ∈Aˉ\xi \in \bar Aξ∈Aˉ and η∈Bˉ\eta \in \bar Bη∈Bˉ if and only if

ξτ1=0 whenever ∑τ2H(τ1,τ2)ητ2<max⁡τ1′∑τ2H(τ1′,τ2)ητ2,\xi_{\tau_1} = 0 \text{ whenever } \sum_{\tau_2} \mathcal H(\tau_1, \tau_2)\eta_{\tau_2} < \max_{\tau_1'} \sum_{\tau_2} \mathcal H(\tau_1', \tau_2)\eta_{\tau_2},ξτ1​​=0 whenever τ2​∑​H(τ1​,τ2​)ητ2​​<τ1′​max​τ2​∑​H(τ1′​,τ2​)ητ2​​, ητ2=0 whenever ∑τ1H(τ1,τ2)ξτ1>min⁡τ2′∑τ1H(τ1,τ2′)ξτ1.\eta_{\tau_2} = 0 \text{ whenever } \sum_{\tau_1} \mathcal H(\tau_1, \tau_2)\xi_{\tau_1} > \min_{\tau_2'} \sum_{\tau_1} \mathcal H(\tau_1, \tau_2')\xi_{\tau_1}.ητ2​​=0 whenever τ1​∑​H(τ1​,τ2​)ξτ1​​>τ2′​min​τ1​∑​H(τ1​,τ2′​)ξτ1​​.

The statement fixes no value and no constant; it says which pairs of mixed strategies are optimal.

Milestones, in attack order

  1. (13:A*) Max⁡xMin⁡yϕ≤Min⁡yMax⁡xϕ\operatorname{Max}_x \operatorname{Min}_y \phi \le \operatorname{Min}_y \operatorname{Max}_x \phiMaxx​Miny​ϕ≤Miny​Maxx​ϕ.
  2. (13:D*) If Max⁡Min⁡=Min⁡Max⁡\operatorname{Max}\operatorname{Min} = \operatorname{Min}\operatorname{Max}MaxMin=MinMax, the saddle points of ϕ\phiϕ are exactly Aϕ×BϕA^\phi \times B^\phiAϕ×Bϕ.
  3. (17:A) Min⁡ηK(ξ,η)=Min⁡τ2∑τ1H(τ1,τ2)ξτ1\operatorname{Min}_\eta K(\xi, \eta) = \operatorname{Min}_{\tau_2} \sum_{\tau_1} \mathcal H(\tau_1, \tau_2)\xi_{\tau_1}Minη​K(ξ,η)=Minτ2​​∑τ1​​H(τ1​,τ2​)ξτ1​​, and dually for Max⁡ξ\operatorname{Max}_\xiMaxξ​.
  4. (16:C) For every matrix a(i,j)a(i, j)a(i,j) exactly one of: some x∈Smx \in S_mx∈Sm​ with ∑ja(i,j)xj≤0\sum_j a(i,j)x_j \le 0∑j​a(i,j)xj​≤0 for all iii; some w∈Snw \in S_nw∈Sn​ with ∑ia(i,j)wi>0\sum_i a(i,j)w_i > 0∑i​a(i,j)wi​>0 for all jjj.
  5. (16:F) The weak form with ≥0\ge 0≥0 in place of >0> 0>0.
  6. (17:6) The minimax theorem: a saddle point of KKK exists (already on the platform as AGT.zero_sum_minimax, proved).
  7. (17:C:f) ξ∈Aˉ\xi \in \bar Aξ∈Aˉ and η∈Bˉ\eta \in \bar Bη∈Bˉ iff ξ,η\xi, \etaξ,η is a saddle point of KKK.

After the goal: (17:E) the game is specially strictly determined iff each player has a pure good strategy.

Significance

(17:D) is the complementary-slackness description of the optimal strategy pairs of a matrix game: a good strategy puts weight only on pure strategies that are best replies to the opponent's good strategy, and conversely any pair of mutually supported best replies is optimal. It is the basis of support-enumeration methods for matrix games, of the equalizing arguments used to solve small games by hand (the book's Chapter IV applies it to Matching Pennies, Stone–Paper–Scissors and Poker), and of the rectangular structure Aˉ×Bˉ\bar A \times \bar BAˉ×Bˉ of the set of optimal pairs. (17:E) connects the mixed-strategy solution to the pure-strategy theory of §14 and to the perfect-information games of §15.

The results are classical and proved in the book. The minimax theorem itself is already machine-checked on the platform (AGT.zero_sum_minimax), and Mathlib contains Sion's minimax theorem and the basic saddle-point lemmas for extended-real functions on sets. This mission adds the book's own chain: the §13 saddle-point calculus under its standing attainment hypothesis, the theorems of the alternative (16:C) and (16:F) in the simplex-normalized form the book uses, the reduction (17:A) to pure strategies, and the characterizations (17:C:f), (17:D), (17:E) of good strategies, which are not on the platform in any form.

Difficulty

The "if" direction of (17:D) cannot be proved from the support conditions alone by local reasoning: that a pair of mutual best replies consists of good strategies uses that the value Max⁡ξMin⁡ηK\operatorname{Max}_\xi \operatorname{Min}_\eta KMaxξ​Minη​K equals Min⁡ηMax⁡ξK\operatorname{Min}_\eta \operatorname{Max}_\xi KMinη​Maxξ​K, i.e. the minimax theorem. Without that equality the "if" direction of (13:D*) fails (points of Aϕ×BϕA^\phi \times B^\phiAϕ×Bϕ exist but are not saddle points), so the calculus of §13 alone does not suffice. Likewise (16:C) is not a direct instance of the Farkas lemma forms on the platform: its alternatives are normalized to the simplex and the second one is strict, and both the existence and the mutual exclusion must be shown.

Formalization scope

Lean conventions, fixed throughout:

  • Pure strategies are Fin β₁, Fin β₂ (numbered from 000), the matrix is H : Fin β₁ → Fin β₂ → ℝ, and SβS_\betaSβ​ is Mathlib's stdSimplex ℝ (Fin β).
  • Nonempty strategy sets (β≥1\beta \ge 1β≥1, from "τ = 1, …, β" in 14.1.1): every theorem assumes 0 < β₁, 0 < β₂, or mixed strategies ξ∈Sβ1\xi \in S_{\beta_1}ξ∈Sβ1​​, η∈Sβ2\eta \in S_{\beta_2}η∈Sβ2​​, which force it. The theorems of the alternative assume n,m≥1n, m \ge 1n,m≥1 (a matrix with rows and columns).
  • Standing hypothesis of 13.2.1 ("we are restricting our considerations to such functions, for which Max and Min exist"): the §13 results (13:A*), (13:D*) are stated for an arbitrary ϕ:X×Y→R\phi : X \times Y \to \mathbb Rϕ:X×Y→R under the predicate MaxMinAttained φ, which says that Min⁡yϕ(x,y)\operatorname{Min}_y \phi(x, y)Miny​ϕ(x,y), Max⁡xϕ(x,y)\operatorname{Max}_x \phi(x, y)Maxx​ϕ(x,y), Max⁡xMin⁡yϕ\operatorname{Max}_x \operatorname{Min}_y \phiMaxx​Miny​ϕ and Min⁡yMax⁡xϕ\operatorname{Min}_y \operatorname{Max}_x \phiMiny​Maxx​ϕ are attained. (13:D*) also carries the hypothesis of 13.5.2 that saddle points exist, stated as Max⁡xMin⁡yϕ=Min⁡yMax⁡xϕ\operatorname{Max}_x \operatorname{Min}_y \phi = \operatorname{Min}_y \operatorname{Max}_x \phiMaxx​Miny​ϕ=Miny​Maxx​ϕ.
  • Max⁡\operatorname{Max}Max and Min⁡\operatorname{Min}Min are the real ⨆, ⨅; they are the book's attained values under the hypotheses above (compactness of the simplex and continuity of KKK for the mixed game). (17:A) asserts attainment explicitly (IsLeast, IsGreatest). "Does not assume its maximum at τ1\tau_1τ1​" in the goal is written without any Max operator.
  • Aˉ\bar AAˉ, Bˉ\bar BBˉ are defined as maximizers and minimizers directly from KKK, not through an assumed value v′v'v′.

A trivializing formalization is ruled out: Aˉ\bar AAˉ and Bˉ\bar BBˉ are not taken as hypotheses or defined through the support conditions, strategy sets cannot be empty, and no Max over an empty or unbounded set occurs.

Contributions welcome: proofs of the milestones, especially (16:C) (from Mathlib's convex separation or from a platform Farkas lemma) and the bridge from AGT.zero_sum_minimax to (17:C:f). The §13 lemmas and the (17:A) reduction are reusable by any mission about matrix games or bilinear saddle points.

Selected references

  • J. von Neumann and O. Morgenstern, Theory of Games and Economic Behavior, 60th-anniversary edition, Princeton University Press, 2007 (reprint of the 3rd edition, 1953), §§13, 16, 17. https://doi.org/10.1515/9781400829460
  • J. von Neumann, "Zur Theorie der Gesellschaftsspiele", Mathematische Annalen 100 (1928), 295–320. https://doi.org/10.1007/BF01448847
  • J. Ville, "Sur la théorie générale des jeux où intervient l'habileté des joueurs", in É. Borel, Traité du calcul des probabilités et de ses applications, IV.2, Gauthier-Villars, 1938, 105–113.
  • S. Kakutani, "A generalization of Brouwer's fixed point theorem", Duke Mathematical Journal 8 (1941), 457–459. https://doi.org/10.1215/S0012-7094-41-00838-4
  • D. Gale, H. W. Kuhn and A. W. Tucker, "Linear programming and the theory of games", in Activity Analysis of Production and Allocation, Wiley, 1951, 317–329.
11 thms4 active usersReviewed
Convex OptimizationOptimizationProbability·Captain: mikedeng1

Numerical Techniques for Stochastic Optimization II: Logarithmic Concavity of Probabilistic ConstraintsTextbook

Motivation

Many engineering and economic planning problems must meet random requirements with a prescribed reliability: a reservoir must satisfy demand with probability at least 0.950.950.95, a power system must cover load except on rare days, an inventory must avoid shortage with high probability. Probabilistic constrained programming (also called chance-constrained programming) models this by requiring that a system of random inequalities hold jointly with probability at least ppp.

The first obstacle to solving such problems is structural. The probability that random constraints are satisfied is, in general, neither concave nor convex in the decision, so the feasible set need not be convex and local search can stall. Prékopa's theory of logarithmically concave measures (1971–1973) removed that obstacle for a large class of distributions, and it is the basis of the numerical methods of Chapter 5 of Ermoliev and Wets, Numerical Techniques for Stochastic Optimization (Springer 1988). This mission formalizes the structural theorems of that chapter.

Timeline:

  • 1959: Charnes and Cooper, individual chance constraints.
  • 1971: Prékopa, logarithmic concave measures with application to stochastic programming (Acta Sci. Math. Szeged 32).
  • 1973: Prékopa, logarithmic concave measures and functions (Acta Sci. Math. Szeged 34), containing the marginal theorem: marginals of log-concave functions are log-concave.
  • 1970s–1980s: nonlinear programming methods for (5.1) (SUMT with logarithmic penalty, supporting hyperplanes, reduced gradients) combined with Monte Carlo evaluation of h0h_0h0​; the chapter surveys them.

Setting

Let ξ\xiξ be a random vector in Rq\mathbb R^qRq on a probability space (Ω,F,P)(\Omega, \mathcal F, P)(Ω,F,P) and let g1,…,gr:Rn×Rq→Rg_1, \dots, g_r : \mathbb R^n \times \mathbb R^q \to \mathbb Rg1​,…,gr​:Rn×Rq→R. The chapter studies problem (5.1):

min⁡h(x)s.t.h0(x)=P(g1(x,ξ)≥0,…,gr(x,ξ)≥0)≥p,h1(x)≥p1,…,hm(x)≥pm.\min h(x) \quad \text{s.t.} \quad h_0(x) = P\bigl(g_1(x,\xi) \ge 0, \dots, g_r(x,\xi) \ge 0\bigr) \ge p, \quad h_1(x) \ge p_1, \dots, h_m(x) \ge p_m .minh(x)s.t.h0​(x)=P(g1​(x,ξ)≥0,…,gr​(x,ξ)≥0)≥p,h1​(x)≥p1​,…,hm​(x)≥pm​.

The function h0h_0h0​ is the probability function (chanceProb in Lean). In the special case gi(x,y)=Tix−yig_i(x,y) = T_i x - y_igi​(x,y)=Ti​x−yi​ it equals F(Tx)F(Tx)F(Tx), where FFF is the joint distribution function of ξ\xiξ.

A function f≥0f \ge 0f≥0 is logarithmically concave on a convex set SSS if

f(λu+(1−λ)v)≥f(u)λf(v)1−λ,u,v∈S, 0<λ<1.f(\lambda u + (1-\lambda) v) \ge f(u)^{\lambda} f(v)^{1-\lambda}, \qquad u, v \in S,\ 0 < \lambda < 1 .f(λu+(1−λ)v)≥f(u)λf(v)1−λ,u,v∈S, 0<λ<1.

Where f>0f > 0f>0 this is concavity of log⁡f\log flogf; the power form also makes sense where f=0f = 0f=0. Nondegenerate normal densities, uniform densities on convex bodies and exponential densities are log-concave.

Section 5.7 introduces the polynomial distribution (5.19) on the cube 0<zj≤10 < z_j \le 10<zj​≤1:

F(z1,…,zn)=1∑i=1Nciz1αi1⋯znαin,ci>0, αij≤0, ∑jαij<0F(z_1, \dots, z_n) = \frac{1}{\sum_{i=1}^{N} c_i z_1^{\alpha_{i1}} \cdots z_n^{\alpha_{in}}}, \qquad c_i > 0,\ \alpha_{ij} \le 0,\ \textstyle\sum_j \alpha_{ij} < 0F(z1​,…,zn​)=∑i=1N​ci​z1αi1​​⋯znαin​​1​,ci​>0, αij​≤0, ∑j​αij​<0

(polyDistF in Lean).

Formalization targets

Goal: Theorem 5.1

If g1,…,grg_1, \dots, g_rg1​,…,gr​ are jointly concave on Rn+q\mathbb R^{n+q}Rn+q and ξ\xiξ has a log-concave density fff on Rq\mathbb R^qRq, then

h0 is logarithmically concave on Rn.h_0 \text{ is logarithmically concave on } \mathbb R^n .h0​ is logarithmically concave on Rn.

Its immediate consequence is that the feasible set {x:h0(x)≥p}\{x : h_0(x) \ge p\}{x:h0​(x)≥p} is convex for every ppp.

Milestones

  1. Theorem 5.2.1: if hhh is log-concave on the convex set H={h≥p}H = \{h \ge p\}H={h≥p}, 0<p<10 < p < 10<p<1, then h−ph - ph−p is log-concave on HHH. This makes the logarithmic penalty function (5.5) of the SUMT method convex.
  2. Theorem 5.2.2: under the standing assumptions of §5.2, every interior point zzz of the feasible set of (5.1) satisfies hi(z)>pih_i(z) > p_ihi​(z)>pi​, i=0,…,mi = 0, \dots, mi=0,…,m.
  3. Theorem 5.7.1 (proved content, (5.21)): for n=2n = 2n=2 and oppositely ordered exponents, ∂2F/∂z1∂z2≥0\partial^2 F / \partial z_1 \partial z_2 \ge 0∂2F/∂z1​∂z2​≥0 on (0,1)2(0,1)^2(0,1)2.
  4. Theorem 5.7.2: the polynomial distribution function is log-concave on (0,1]n(0,1]^n(0,1]n.

The platform theorem ConvexOptimization.prekopa_marginal_log_concave (Prékopa's marginal theorem, proved) is included as a reference item.

Significance

Theorem 5.1 turns a probabilistic constraint into a convex constraint after taking logarithms. This is what makes convergence proofs for nonlinear programming methods (SUMT with logarithmic penalty, supporting hyperplanes, reduced gradients) applicable to (5.1) and to the reliability maximization problem (5.4); without it, those methods have no guarantee of finding a global optimum. Theorems 5.2.1 and 5.2.2 are the two facts that make the SUMT method of §5.2 well defined and convex on the interior of the feasible set. Theorem 5.7.2 shows that probabilistic constraints under the polynomial distribution define convex sets, so they can be added to geometric programmes.

On status: Theorem 5.1 is a classical result, proved in Prékopa's papers (the chapter itself refers to Prékopa's survey for the proof). Prékopa's marginal theorem and the Prékopa–Leindler inequality are already machine-checked on this platform; Theorem 5.1 and the §5.2 and §5.7 theorems are, as far as a search of the platform shows, not formalized. The work is formalizing known proofs, in the log-concavity predicate the platform already uses.

Difficulty

The obvious argument for Theorem 5.1, "the constraint set is convex and the density is log-concave, so the probability is log-concave", hides the real content: log-concavity of a probability as a function of a parameter is a statement about integrals, and it does not follow from pointwise properties of the integrand without a Prékopa–Leindler-type inequality. Concavity of each gig_igi​ separately in xxx and in yyy is not enough; joint concavity in (x,y)(x, y)(x,y) is used essentially. The probability function vanishes on large regions in typical examples, so any argument that takes logarithms pointwise fails at the boundary of its support.

For Theorem 5.2.2 the naive argument fails at the index i=0i = 0i=0: nothing about h0h_0h0​ is assumed directly, and log-concavity of h0h_0h0​ is exactly Theorem 5.1. For Theorem 5.7.1 the difficulty is a sign condition on a covariance; without the ordering hypothesis the mixed derivative can be negative.

Formalization scope

Rn\mathbb R^nRn and Rq\mathbb R^qRq are EuclideanSpace ℝ (Fin n) and EuclideanSpace ℝ (Fin q); the constraint functions take pairs (x,y)(x, y)(x,y) in the product, and concavity is ConcaveOn ℝ Set.univ on that product (joint concavity). A "continuous probability distribution with density fff" is stated as P.map ξ = volume.withDensity (ENNReal.ofReal ∘ f) with ξ\xiξ and fff measurable and PPP a probability measure. Log-concavity is the platform definition ConvexOptimization.LogConcaveOn (nonnegativity plus the power inequality), imported as a reference item; concavity of Real.log ∘ h₀ would be a different, wrong property because Lean's Real.log 0 = 0. Indices are 0-based (Fin r, Fin m, Fin N, Fin n); the probabilistic constraint i=0i = 0i=0 of Theorem 5.2.2 is stated separately from h1,…,hmh_1, \dots, h_mh1​,…,hm​. The polynomial distribution is a formula on Fin n → ℝ with real powers, used only on the cube.

No constant of the book is replaced by an explicit value: every result of this chapter is qualitative.

Corrections of the printed text, recorded in each item's Formalization Note:

  • Theorem 5.1 states the density condition "for every x1,x2∈Rnx_1, x_2 \in \mathbb R^nx1​,x2​∈Rn"; the density lives on Rq\mathbb R^qRq and the condition is imposed there.
  • (5.19) prints the first factor as ziαi1z_i^{\alpha_{i1}}ziαi1​​ and the domain index as i=1,…,Ni = 1, \dots, Ni=1,…,N; they are read as z1αi1z_1^{\alpha_{i1}}z1αi1​​ and j=1,…,nj = 1, \dots, nj=1,…,n.
  • Theorem 5.7.1 prints its ordering hypothesis with transposed indices (α11≤α12≤⋯≤α1n\alpha_{11} \le \alpha_{12} \le \dots \le \alpha_{1n}α11​≤α12​≤⋯≤α1n​); following the proof, it is read as: across the NNN terms the z1z_1z1​-exponents increase and the z2z_2z2​-exponents decrease.
  • Theorem 5.7.1 claims "is a probability distribution function"; the book proves only (5.21), and normalization would need ∑ici=1\sum_i c_i = 1∑i​ci​=1, which is not assumed. The formal statement is (5.21), the mixed derivative as an iterated deriv.

Theorem 5.2.2 carries all six assumptions of §5.2, including compactness of the feasible set and the Slater point; it does not assume log-concavity of h0h_0h0​, which must be derived from the density and concavity hypotheses. A trivializing formalization, such as one with a density hypothesis that no probability law satisfies or a log-concavity predicate that holds for every function vanishing somewhere, is ruled out: the density hypotheses are satisfiable (checked locally) and LogConcaveOn is the multiplicative inequality at every pair of points.

Needed infrastructure: Prékopa's marginal theorem (available), log-concavity of indicators of convex sets and of products, measurability of the constraint set, and Artin's theorem that a sum of log-convex functions is log-convex (for Theorem 5.7.2). Artin's theorem and closure properties of LogConcaveOn are reusable beyond this mission, and contributions of them are welcome.

Selected references

  • A. Prékopa, "Numerical Solution of Probabilistic Constrained Programming Problems", in Yu. Ermoliev and R. J-B Wets (eds.), Numerical Techniques for Stochastic Optimization, Springer Series in Computational Mathematics 10, Springer 1988, Ch. 5, pp. 123–139. https://doi.org/10.1007/978-3-642-61370-8
  • A. Prékopa, "Logarithmic concave measures with application to stochastic programming", Acta Sci. Math. (Szeged) 32 (1971), 301–316.
  • A. Prékopa, "On logarithmic concave measures and functions", Acta Sci. Math. (Szeged) 34 (1973), 335–343.
  • A. Charnes and W. W. Cooper, "Chance-constrained programming", Management Science 6 (1959), 73–79. https://doi.org/10.1287/mnsc.6.1.73
  • B. L. Miller and H. M. Wagner, "Chance constrained programming with joint constraints", Operations Research 13 (1965), 930–945. https://doi.org/10.1287/opre.13.6.930
  • A. Prékopa, Stochastic Programming, Kluwer 1995. https://doi.org/10.1007/978-94-017-3087-7
9 thms4 active usersReviewed
🏆Completed
Convex OptimizationLinear OptimizationOptimization+1·Captain: mikedeng1

Numerical Techniques for Stochastic Optimization I: Edmundson–Madansky Bounds for Independent Random Data and Simple RecourseTextbook

Motivation

In a two-stage stochastic linear program a decision xxx is taken before random data ξ\xiξ are observed, and a corrective recourse decision yyy is taken afterwards at a cost. The objective contains the expectation of an optimal value of a linear program, ∫Q(x,ξ(ω)) P(dω)\int Q(x,\xi(\omega))\,P(d\omega)∫Q(x,ξ(ω))P(dω), and for continuous or high-dimensional ξ\xiξ that integral cannot be evaluated exactly. Practical methods therefore replace ξ\xiξ by a discrete random vector and control the error by computable lower and upper bounds on the expected recourse cost. Chapter 2 of Ermoliev and Wets (eds.), Numerical Techniques for Stochastic Optimization (Springer 1988), by P. Kall, A. Ruszczyński and K. Frauendorfer, surveys these bounds as they were used in the codes of the time: Jensen's inequality from below, the Edmundson–Madansky inequality from above, and the special structure of simple recourse, where the expected cost is available in closed form.

Timeline. Jensen's inequality (1906) gives the lower bound for a convex integrand. A. Madansky, "Bounds on the expectation of a convex function of a multivariate random variable", Ann. Math. Statist. 30 (1959), and H. P. Edmundson (RAND report, 1956) gave the upper bound by the two-point law on the endpoints of an interval, and its product version for independent components. Kall and Stoyan (1982), Huang, Ziemba and Ben-Tal (1977), Frauendorfer and Kall (1988) developed partition refinement of both bounds, the scheme this chapter describes; Frauendorfer (1988) extended the upper bound to dependent data on boxes.

Setting

The two-stage problem (2.11) is: minimize ψ(x)=cTx+∫ΩQ(x,ξ(ω)) P(dω)\psi(x)=c^Tx+\int_\Omega Q(x,\xi(\omega))\,P(d\omega)ψ(x)=cTx+∫Ω​Q(x,ξ(ω))P(dω) subject to Ax=bAx=bAx=b, x≥0x\ge 0x≥0. The recourse cost Q(x,ξ)Q(x,\xi)Q(x,ξ) is the optimal value of the second-stage problem (2.12),

Q(x,ξ)=min⁡{qTy:Wy=h−Tx, y≥0},ξ=(q,h,T),Q(x,\xi)=\min\{q^Ty : Wy=h-Tx,\ y\ge 0\},\qquad \xi=(q,h,T),Q(x,ξ)=min{qTy:Wy=h−Tx, y≥0},ξ=(q,h,T),

with a deterministic m2×n2m_2\times n_2m2​×n2​ matrix WWW (fixed recourse), and Q=+∞Q=+\inftyQ=+∞ when (2.12) is infeasible. Throughout the chapter the book assumes complete recourse, {Wy:y≥0}=Rm2\{Wy:y\ge0\}=\mathbb R^{m_2}{Wy:y≥0}=Rm2​, and dual feasibility: for every realization of qqq some uuu satisfies WTu≤qW^Tu\le qWTu≤q. Under these assumptions QQQ is finite. The expected recourse function is Q(x)=∫Q(x,ξ(ω)) P(dω)\mathcal Q(x)=\int Q(x,\xi(\omega))\,P(d\omega)Q(x)=∫Q(x,ξ(ω))P(dω).

The Edmundson–Madansky law of an interval [a,b][a,b][a,b], a<ba<ba<b, with mean ξ0\xi^0ξ0 puts mass p1=(b−ξ0)/(b−a)p_1=(b-\xi^0)/(b-a)p1​=(b−ξ0)/(b−a) at aaa and p2=(ξ0−a)/(b−a)p_2=(\xi^0-a)/(b-a)p2​=(ξ0−a)/(b−a) at bbb (2.32). For a box Ξ=×j=1m[aj,bj]\Xi=\times_{j=1}^m[a_j,b_j]Ξ=×j=1m​[aj​,bj​] and means ξj0\xi^0_jξj0​, the vector ξ^\hat\xiξ^​ with independent components of these two-point laws sits at the vertex vvv with probability ∏jpj(vj)\prod_j p_j(v_j)∏j​pj​(vj​).

Simple recourse is the case W=[I,−I]W=[I,-I]W=[I,−I], q=[q+,q−]q=[q^+,q^-]q=[q+,q−] with qj++qj−≥0q^+_j+q^-_j\ge0qj+​+qj−​≥0, deterministic TTT and random hhh only. With χ=Tx\chi=Txχ=Tx the recourse cost splits into one-row costs Qj(χj,hj)=qj+(hj−χj)Q_j(\chi_j,h_j)=q^+_j(h_j-\chi_j)Qj​(χj​,hj​)=qj+​(hj​−χj​) if hj≥χjh_j\ge\chi_jhj​≥χj​, and qj−(χj−hj)q^-_j(\chi_j-h_j)qj−​(χj​−hj​) otherwise.

Formalization targets

Goal: the Edmundson–Madansky bound for independent components (p. 46)

If ξ\xiξ has independent components ξj∈[aj,bj]\xi_j\in[a_j,b_j]ξj​∈[aj​,bj​] with means ξj0\xi^0_jξj0​, and φ\varphiφ is convex on Ξ=×j[aj,bj]\Xi=\times_j[a_j,b_j]Ξ=×j​[aj​,bj​], then

Eφ(ξ)≤∑v∈vert Ξ(∏j=1mpj(vj))φ(v).E\varphi(\xi)\le\sum_{v\in\mathrm{vert}\,\Xi}\Big(\prod_{j=1}^m p_j(v_j)\Big)\varphi(v).Eφ(ξ)≤v∈vertΞ∑​(j=1∏m​pj​(vj​))φ(v).

The book applies it to φ=Q(x,⋅)\varphi=Q(x,\cdot)φ=Q(x,⋅); the goal is stated for every convex φ\varphiφ, with the explicit weights of (2.32).

Milestones

  1. Properties (b), (d), (e) of p. 40: Q(x,⋅)Q(x,\cdot)Q(x,⋅) is piecewise linear and convex in (h,T)(h,T)(h,T); Q(⋅,ξ)Q(\cdot,\xi)Q(⋅,ξ) is convex piecewise linear in xxx; the expected recourse function is finite and convex under finite second moments.
  2. The Jensen lower bound (2.26)–(2.27) on a partition (a published, proved theorem, reused).
  3. The dual-multiplier lower bound (2.30)–(2.31).
  4. The one-dimensional Edmundson–Madansky inequality (2.32)–(2.34).
  5. For simple recourse: separability (2.46)–(2.49), the closed form (2.51) of EQjEQ_jEQj​, and the bounds (2.55)–(2.56) from the one-block problem.

Significance

The upper bound is the half of the bounding scheme that is not automatic. Jensen's inequality needs only a mean; an upper bound on the expectation of a convex function needs a bounded support and, in the product form, independence. Together they give a certified interval for the optimal value of a two-stage problem, and repeated partitioning of the support shrinks that interval; this is the basis of the sequential approximation methods of §2.2.4 and of later codes. The dual-multiplier bound and the simple-recourse formulas are the pieces that make those intervals cheap to compute.

The results are classical and proved in the literature cited on the page. The one-dimensional Edmundson–Madansky inequality and the general extreme-point form of the upper bound (a measure on the extreme points reproducing the barycentre) are already formalized on Prove2Me in the Introduction to Stochastic Programming series, as is the partition Jensen bound. The product form for independent components is not: deriving it from the extreme-point form requires constructing the product kernel, which is the content of this mission. The recourse properties (b), (d), (e) for a general distribution with finite second moments, the dual-multiplier bound and the simple-recourse formulas are not formalized anywhere known to this mission.

Difficulty

The obvious argument inducts on the dimension, applying the one-dimensional inequality in one coordinate while the others are held fixed. That step needs the conditional law of the remaining coordinates given the first to be their unconditional law, i.e. independence expressed as a product decomposition of the joint law, and it needs φ\varphiφ with one coordinate replaced by an endpoint to remain convex on the lower-dimensional box and integrable. For dependent components the inequality is false with these weights: on [0,1]2[0,1]^2[0,1]2 with means (12,12)(\tfrac12,\tfrac12)(21​,21​) and φ(x,y)=(x−y)2\varphi(x,y)=(x-y)^2φ(x,y)=(x−y)2, the product law gives 12\tfrac1221​ while mass 12\tfrac1221​ at (1,0)(1,0)(1,0) and at (0,1)(0,1)(0,1) gives 111. The book's remark that the product law is extremal among all laws on Ξ\XiΞ with the given mean fails for this reason when m≥2m\ge2m≥2, and is not part of this mission.

For the recourse properties the difficulty is bookkeeping: QQQ is an extended-real optimal value, and finiteness, measurability in ω\omegaω and integrability must be derived from complete recourse, dual feasibility and the moment hypothesis rather than assumed.

Formalization scope

Vectors are functions from finite index types to R\mathbb RR (ι → ℝ), matrices are Mathlib Matrix, and random data live on a probability space (Ω, P). The recourse cost is an EReal infimum over the feasible set, so infeasibility gives +∞+\infty+∞ and unboundedness −∞-\infty−∞ exactly as on p. 39; theorems that integrate it carry complete recourse and dual feasibility, which make it finite. The expected recourse function integrates the real part of the recourse cost. Independence of the components is ProbabilityTheory.iIndepFun; the box is Set.pi univ (fun j => Icc (a j) (b j)), with aj<bja_j<b_jaj​<bj​, and values in the box are required almost surely. The upper bound is the explicit sum over Boolean vertex labels of products of the weights (2.32); no abstract extremal measure is used.

Conventions fixed where the page is silent or ambiguous:

  • Properties (b), (d), (e) are stated on all of Rn1\mathbb R^{n_1}Rn1​: under the standing complete-recourse assumption K2=Rn1K_2=\mathbb R^{n_1}K2​=Rn1​. "Convex piecewise linear" is rendered as a maximum of finitely many affine functions.
  • The book's hypothesis of finite second moments in (e) is kept as stated, componentwise.
  • The book writes QQQ for both Q(x,ξ)Q(x,\xi)Q(x,ξ) and Q(x)\mathcal Q(x)Q(x), and reuses Q~\tilde QQ~​, ψ~\tilde\psiψ~​ for different functions in (2.27) and (2.30)–(2.31); the Lean names are recourseCost, expectedRecourse and dualLowerBound.
  • In (2.51) a conditional mean on a null event is 000 in Lean; it always appears multiplied by that event's probability, so the formula is unchanged.
  • In (2.56) the minimum is a real infimum over the nonempty first-stage feasible set; attainment is not claimed.
  • No constant of the chapter is hidden behind O(⋅)O(\cdot)O(⋅); all bounds are explicit.

A goal stated for affine φ\varphiφ (where it is an equality), or with ξ^\hat\xiξ^​ allowed to be any discrete law with the right mean, would be trivial or a different theorem; the weights are the products of (2.32), and independence of the components is a hypothesis.

A complete development needs: finite-dimensional LP duality with extended-real values (reusable across all recourse missions), measurability and integrability of optimal-value functions, the conditional-independence step for product measures, and the one-dimensional chord inequality. The partitioned upper bound (2.37) and the discrete reformulation (2.21), (2.28) are natural follow-up statements on the same definitions.

Selected references

  • P. Kall, A. Ruszczyński, K. Frauendorfer, "Approximation Techniques in Stochastic Programming", in Yu. Ermoliev and R. J-B Wets (eds.), Numerical Techniques for Stochastic Optimization, Springer Series in Computational Mathematics 10, Springer 1988, Ch. 2, pp. 33–64. https://doi.org/10.1007/978-3-642-61370-8
  • A. Madansky, "Bounds on the expectation of a convex function of a multivariate random variable", Annals of Mathematical Statistics 30 (1959), 743–746. https://doi.org/10.1214/aoms/1177706203
  • P. Kall, Stochastic Linear Programming, Springer 1976. https://doi.org/10.1007/978-3-642-66252-2
  • R. J-B Wets, "Stochastic programs with fixed recourse: the equivalent deterministic program", SIAM Review 16 (1974), 309–339. https://doi.org/10.1137/1016053
  • K. Frauendorfer, "Solving SLP recourse problems with arbitrary multivariate distributions — the dependent case", Mathematics of Operations Research 13 (1988), 377–394. https://doi.org/10.1287/moor.13.3.377
  • J. R. Birge, F. Louveaux, Introduction to Stochastic Programming, Springer 1997, Ch. 8. https://doi.org/10.1007/b97617
13 thms4 active usersReviewed
Convex OptimizationOptimizationProbability·Captain: mikedeng1

Introduction to the Scenario Approach I: The Violation Distribution of the Scenario SolutionTextbook

Motivation

Many design problems in control, finance and operations research are convex programs with uncertain constraints: a decision θ\thetaθ must satisfy θ∈Θδ\theta\in\Theta_\deltaθ∈Θδ​ for a parameter δ\deltaδ that is not known in advance. Enforcing the constraint for every possible δ\deltaδ (robust optimization) is often intractable or too conservative, and a chance-constrained formulation needs the distribution of δ\deltaδ, which in practice is rarely known. The scenario approach replaces the uncertain constraint by the constraints of NNN observed samples δ1,…,δN\delta_1,\dots,\delta_Nδ1​,…,δN​ and solves the resulting ordinary convex program. The question it answers is how much of the unseen uncertainty the resulting decision still violates.

The answer, the generalization theorem of the scenario approach, is distribution-free: the probability that the scenario solution violates more than a fraction ε\varepsilonε of the uncertainty is bounded by a binomial tail that depends only on NNN and on the number ddd of decision variables. This mission formalizes that theorem as it is presented in Chapters 3 and 5 of Campi and Garatti's textbook Introduction to the Scenario Approach (SIAM/MOS 2018), the first mission of a series on the book.

Timeline. Calafiore and Campi introduced scenario programs and bounded the violation of their solutions through the count of support constraints (Math. Program. 2005; IEEE TAC 2006). Campi and Garatti proved in 2008 that the binomial-tail bound of Theorem 3.7 holds for every convex scenario program under existence and uniqueness of the solution, and that it is attained with equality by fully supported problems (SIAM J. Optim. 2008), which settled the tightness question. The textbook (DOI 10.1137/1.9781611975444) presents the theorem with a complete proof for fully supported problems in the plane.

Setting

Fix a cost vector c∈Rdc\in\mathbb R^dc∈Rd, a domain Θ⊆Rd\Theta\subseteq\mathbb R^dΘ⊆Rd, a measurable space Δ\DeltaΔ of uncertainty instances with a probability P\mathbb PP, and a constraint set Θδ⊆Rd\Theta_\delta\subseteq\mathbb R^dΘδ​⊆Rd for each δ∈Δ\delta\in\Deltaδ∈Δ.

  • The violation of a decision θ\thetaθ (Definition 3.1) is V(θ)=P{δ∈Δ:θ∉Θδ}V(\theta)=\mathbb P\{\delta\in\Delta:\theta\notin\Theta_\delta\}V(θ)=P{δ∈Δ:θ∈/Θδ​}, the probability that θ\thetaθ fails the constraint of a fresh instance.
  • For a sample (δ1,…,δm)(\delta_1,\dots,\delta_m)(δ1​,…,δm​), the scenario program is
min⁡θ∈ΘcTθsubject toθ∈⋂i=1mΘδi.\min_{\theta\in\Theta}c^T\theta\quad\text{subject to}\quad\theta\in\bigcap_{i=1}^{m}\Theta_{\delta_i}.θ∈Θmin​cTθsubject toθ∈i=1⋂m​Θδi​​.

A solution is a feasible point of least cost. With m=Nm=Nm=N i.i.d. samples its solution is denoted θ∗\theta^*θ∗; it is a random vector, a function of the sample, and V(θ∗)V(\theta^*)V(θ∗) is a random variable in [0,1][0,1][0,1].

  • Assumption 3.4 (convexity): Θ\ThetaΘ and every Θδ\Theta_\deltaΘδ​ are convex and closed. Assumption 3.6 (existence and uniqueness): for every mmm and every sample, the program with mmm constraints has exactly one solution.
  • A constraint is a support constraint (Definition 5.1) if its removal improves the solution. A problem is fully supported (Definition 5.4) if for every m≥dm\ge dm≥d the program with mmm constraints has, with probability 1, exactly ddd support constraints.

Formalization targets

Goal: Theorem 3.7

For 1≤d≤N1\le d\le N1≤d≤N and every ε∈[0,1]\varepsilon\in[0,1]ε∈[0,1], under Assumptions 3.4 and 3.6,

PN{V(θ∗)>ε}≤∑i=0d−1(Ni)εi(1−ε)N−i.\mathbb P^N\{V(\theta^*)>\varepsilon\}\le\sum_{i=0}^{d-1}\binom Ni\varepsilon^i(1-\varepsilon)^{N-i}.PN{V(θ∗)>ε}≤i=0∑d−1​(iN​)εi(1−ε)N−i.

The right-hand side is the upper tail of a Beta(d,N−d+1)(d,N-d+1)(d,N−d+1) distribution. The statement leaves P\mathbb PP, Θ\ThetaΘ and the constraint family completely unspecified beyond the two assumptions.

Milestones

  1. Helly's lemma (Lemma 5.3), referenced from the platform in its ddd-dimensional form.
  2. Theorem 5.2: for every mmm and every sample, a convex scenario program has at most ddd support constraints.
  3. Eq. (5.3): for a fully supported problem with d=N=2d=N=2d=N=2, P2{V(θ∗)>ε}=1−ε2\mathbb P^2\{V(\theta^*)>\varepsilon\}=1-\varepsilon^2P2{V(θ∗)>ε}=1−ε2.
  4. Eq. (5.2): for fully supported problems, Theorem 3.7 holds with equality.
  5. Eqs. (3.5)–(3.6): the bound is the Beta(d,N−d+1)(d,N-d+1)(d,N−d+1) distribution function (a published incomplete-beta identity).
  6. Eq. (3.9): ∑i=0d−1(Ni)εi(1−ε)N−i≤2d−1(1−ε/2)N≤2d−1e−εN/2\sum_{i=0}^{d-1}\binom Ni\varepsilon^i(1-\varepsilon)^{N-i}\le 2^{d-1}(1-\varepsilon/2)^N\le2^{d-1}e^{-\varepsilon N/2}∑i=0d−1​(iN​)εi(1−ε)N−i≤2d−1(1−ε/2)N≤2d−1e−εN/2.
  7. Theorem 3.8: E[V(θ∗)]≤d/(N+1)\mathbb E[V(\theta^*)]\le d/(N+1)E[V(θ∗)]≤d/(N+1).
  8. Theorem 1.3: if N≥2ε(ln⁡1β+d−1)N\ge\frac2\varepsilon(\ln\frac1\beta+d-1)N≥ε2​(lnβ1​+d−1), then V(θ∗)≤εV(\theta^*)\le\varepsilonV(θ∗)≤ε with probability at least 1−β1-\beta1−β.

Significance

Theorem 3.7 is what makes the scenario approach usable as a design method: it certifies the reliability of a decision computed from data without any knowledge of the data-generating distribution, requiring only independence of the samples. Theorems 3.8 and 1.3 are its two most used consequences, an expected-violation bound and an explicit sample size, and later chapters of the book (constraint removal, the FAST algorithm, empirical-cost results) build on the same statement. Equality for fully supported problems shows that the bound cannot be improved for any ddd and NNN.

The theorem is proved in the literature; it has no machine-checked proof. A formalization produces, beyond the result itself, a reusable library of scenario programs, violation probabilities and support constraints on which the rest of the series (constraint removal, nonconvex support sets) can be stated, and it checks the measure-theoretic content that the book deliberately leaves aside ("measurability issues are glossed over throughout", p. 33).

Difficulty

The deterministic part, at most ddd support constraints, is a short consequence of Helly's theorem. The probabilistic part is where the obvious approach fails. A uniform-convergence argument over all θ\thetaθ (Vapnik–Chervonenkis theory, footnote 11 of the book) gives bounds of the wrong order and can be vacuous, because it ignores that only the solution matters. The sharp bound is an exact statement about the law of V(θ∗)V(\theta^*)V(θ∗), not a union bound, and problems with fewer than ddd support constraints, or with degenerate configurations of constraints, must be shown to be no worse than fully supported ones; the book treats the general case only in the plane and refers to Campi & Garatti 2008 for general ddd. Handling the null sets and the exchangeability of the samples under the product measure is a substantial part of the work.

Formalization scope

  • Decisions live in EuclideanSpace ℝ (Fin d); the cost is inner ℝ c θ. A sample of size mmm is ω : Fin m → Δ with law Measure.pi (fun _ : Fin m => P), and P is a probability measure.
  • The violation is the real number (P {δ | θ ∉ Θδ δ}).toReal; probabilities of events over the sample are compared in [0,∞][0,\infty][0,∞] through ENNReal.ofReal.
  • The solution θ∗\theta^*θ∗ is a function θstar : (Fin N → Δ) → EuclideanSpace ℝ (Fin d) together with the hypothesis that θstar ω solves the program for every sample; it is never an arbitrary map.
  • Assumption 3.6 is stated for every mmm including m=0m=0m=0 (a unique minimizer on Θ\ThetaΘ itself) and for every sample, as on the page.
  • Hypotheses the page leaves implicit are explicit: the constraint relation {(θ,δ):θ∈Θδ}\{(\theta,\delta):\theta\in\Theta_\delta\}{(θ,δ):θ∈Θδ​} is jointly measurable and θstar is measurable (the book's p. 33 convention); ε∈[0,1]\varepsilon\in[0,1]ε∈[0,1]; d≥1d\ge1d≥1.
  • A support constraint is one whose removal leaves a feasible point of strictly smaller cost than the solution; the count is a Finset.card over Fin m.
  • Theorem 3.8 asserts integrability of V(θ∗)V(\theta^*)V(θ∗) together with the bound, so it cannot hold through the convention that a non-integrable function integrates to 000. Theorem 1.3's own sentence omits the assumptions; they are added as in §3.2.1, where it is derived from Theorem 3.7.

A statement in which θ∗\theta^*θ∗ is any feasible point of the program, or merely a measurable function of the sample, is false and does not count as a formalization of Theorem 3.7; nor does one in which Assumption 3.6 is weakened to almost every sample. Contributions of general infrastructure are welcome: exchangeability arguments for product measures, the binomial–beta identity, and Helly-type counting lemmas are reusable well beyond this mission.

Selected references

  • M. C. Campi, S. Garatti, Introduction to the Scenario Approach, MOS-SIAM Series on Optimization 26, SIAM/MOS, 2018. https://doi.org/10.1137/1.9781611975444
  • M. C. Campi, S. Garatti, The exact feasibility of randomized solutions of uncertain convex programs, SIAM Journal on Optimization 19(3), 2008. https://doi.org/10.1137/07069821X
  • G. Calafiore, M. C. Campi, Uncertain convex programs: randomized solutions and confidence levels, Mathematical Programming 102, 2005. https://doi.org/10.1007/s10107-003-0499-y
  • G. Calafiore, M. C. Campi, The scenario approach to robust control design, IEEE Transactions on Automatic Control 51(5), 2006. https://doi.org/10.1109/TAC.2006.875041
  • E. Helly, Über Mengen konvexer Körper mit gemeinschaftlichen Punkten, Jahresbericht der DMV 32, 1923.
12 thms4 active usersReviewed
🏆Completed
Algorithmic Game TheoryMechanism DesignOptimization·Captain: mikedeng1

Algorithmic Mechanism Design I: MinWork Is a Strongly Truthful n-Approximation Mechanism for Task Scheduling on Unrelated MachinesResearch Paper

Motivation

Algorithmic mechanism design asks for algorithms whose inputs are held by self-interested parties. Each party reports its private data, the algorithm computes an outcome, and payments are arranged so that no party gains by misreporting. Nisan and Ronen introduced the field in Algorithmic Mechanism Design (Games Econ. Behav. 35, 2001). Their running example is task scheduling on unrelated machines: kkk tasks are distributed among nnn machines owned by different agents, each agent knows only its own processing times, and the designer wants to minimize the make-span.

Without incentives the problem is classical: minimizing make-span on unrelated machines is NP-hard and admits a polynomial 2-approximation (Lenstra, Shmoys, Tardos, 1990). With selfish agents the question changes: which approximation ratios can a truthful mechanism guarantee? This mission formalizes the paper's upper bound, the MinWork mechanism, which is the benchmark every later lower bound for truthful scheduling is compared with.

Timeline.

  • 1961: Vickrey introduces the second-price auction (J. Finance 16).
  • 1971–1973: Clarke and Groves generalize it to the VCG family of truthful mechanisms for utilitarian objectives (Groves, Econometrica 41, 1973).
  • 1999/2001: Nisan and Ronen show MinWork is a strongly truthful nnn-approximation, and that no truthful mechanism beats ratio 2.
  • 2007: Christodoulou, Koutsoupias and Vidali raise the deterministic lower bound to 1+21+\sqrt21+2​ for n≥3n \ge 3n≥3; Koutsoupias and Vidali later raise it to 1+φ≈2.6181+\varphi \approx 2.6181+φ≈2.618.
  • 2023: Christodoulou, Koutsoupias and Kovács prove the Nisan–Ronen conjecture: no deterministic truthful mechanism achieves a ratio below nnn (STOC 2023, arXiv:2301.11905), so MinWork is optimal among deterministic truthful mechanisms.

Setting

There are nnn agents and kkk tasks. Agent iii's type is the vector ti=(t1i,…,tki)t^i = (t^i_1,\dots,t^i_k)ti=(t1i​,…,tki​) of positive times, tji>0t^i_j > 0tji​>0 being the time agent iii needs to perform task jjj. A type vector is t=(t1,…,tn)t = (t^1,\dots,t^n)t=(t1,…,tn). An allocation xxx sends each task jjj to one agent; xix^ixi is the set of tasks agent iii receives. The make-span of xxx is

g(x,t)=max⁡i∑j∈xitji,g(x,t) = \max_{i} \sum_{j \in x^i} t^i_j ,g(x,t)=imax​j∈xi∑​tji​,

and agent iii's valuation is vi(x,ti)=−∑j∈xitjiv^i(x,t^i) = -\sum_{j \in x^i} t^i_jvi(x,ti)=−∑j∈xi​tji​.

A direct mechanism asks every agent to declare a type, computes an allocation x(d)x(d)x(d) from the declared vector ddd, and hands agent iii a payment pi(d)p^i(d)pi(d). Agent iii's utility is pi(d)+vi(x(d),ti)p^i(d) + v^i(x(d), t^i)pi(d)+vi(x(d),ti), with tit^iti its true type. The mechanism is truthful if declaring tit^iti maximizes agent iii's utility for every declaration of the others, and strongly truthful if truth-telling is the only such dominant strategy. An allocation rule is a ccc-approximation if g(x(t),t)≤c⋅g(y,t)g(x(t),t) \le c \cdot g(y,t)g(x(t),t)≤c⋅g(y,t) for every type vector ttt and every allocation yyy.

The MinWork mechanism allocates each task to an agent with minimal declared time for it, breaking ties arbitrarily. For each task it wins, an agent receives the second-best declared time min⁡i′≠idji′\min_{i' \ne i} d^{i'}_jmini′=i​dji′​:

pi(d)=∑j∈xi(d)min⁡i′≠idji′.p^i(d) = \sum_{j \in x^i(d)} \min_{i' \neq i} d^{i'}_j .pi(d)=j∈xi(d)∑​i′=imin​dji′​.

The Lean development uses the same names: load, makespan, IsTruthful, IsStronglyTruthful, IsApprox, IsMinWorkAlloc, secondBest, minTime, minWorkPay.

Formalization targets

Goal: Theorem 4.1

For n≥2n \ge 2n≥2 and every MinWork allocation rule xxx with payments ppp as above,

(x,p) is strongly truthfulandg(x(t),t)≤n⋅g(y,t)  for all positive t and all allocations y.(x,p)\ \text{is strongly truthful} \quad\text{and}\quad g(x(t),t) \le n \cdot g(y,t)\ \ \text{for all positive } t \text{ and all allocations } y .(x,p) is strongly truthfulandg(x(t),t)≤n⋅g(y,t)  for all positive t and all allocations y.

Milestones

  1. Theorem 3.1 (Groves): a VGC mechanism is truthful. This is an existing platform theorem, used as a reference.
  2. MinWork belongs to the VGC family. Its allocation maximizes ∑ivi(ti,x)\sum_i v^i(t^i,x)∑i​vi(ti,x), and its payment is ∑i′≠ivi′(ti′,x(t))+h−i\sum_{i'\ne i} v^{i'}(t^{i'},x(t)) + h^{-i}∑i′=i​vi′(ti′,x(t))+h−i with h−i=∑jmin⁡i′≠itji′h^{-i} = \sum_j \min_{i'\ne i} t^{i'}_jh−i=∑j​mini′=i​tji′​.
  3. Claim 4.2: MinWork is strongly truthful.
  4. g(x(t),t)≤∑jmin⁡itjig(x(t),t) \le \sum_{j} \min_i t^i_jg(x(t),t)≤∑j​mini​tji​.
  5. g(y,t)≥1n∑jmin⁡itjig(y,t) \ge \frac1n \sum_j \min_i t^i_jg(y,t)≥n1​∑j​mini​tji​ for every allocation yyy.
  6. Claim 4.3: MinWork is an nnn-approximation.

Significance

The theorem gives the first positive result for truthful scheduling: a mechanism that is truthful in the strongest sense and is within a factor nnn of optimal, whatever the tie-breaking rule. Every lower bound in the paper (Theorems 4.6, 4.10 and 4.12) and in the later literature measures itself against this ratio. Since the 2023 resolution of the Nisan–Ronen conjecture, the ratio nnn is known to be tight for deterministic truthful mechanisms.

The result is proved in the paper; it is not known to be formalized in any proof assistant. The platform already has Groves' theorem in an abstract form (AGT.vcg_incentive_compatible). This mission connects that abstract statement to a concrete combinatorial mechanism, and it adds the strict part of strong truthfulness for any number of tasks and agents, which the paper proves only for one task and two agents. The vocabulary (make-span over unrelated machines, direct scheduling mechanisms, strong truthfulness) is shared with the seven later missions of this series.

Difficulty

Truthfulness follows from Groves' theorem once MinWork is identified as a VGC mechanism. The identification requires the payment identity at every declared vector and under every tie-breaking rule, including ties at the winning time. The main difficulty is the strict part of strong truthfulness. A misreport that differs from the truth only on one task must still be shown to lose strictly for some declarations of the others. Those declarations must stay positive, and on every other task they must leave the outcome unchanged. The paper's proof covers only one task and two agents and leaves the general case as "similar". Its printed inequality also has the two utilities in the wrong order (see below), so it cannot be transcribed directly.

Formalization scope

  • Agents are Fin n and tasks are Fin k. An allocation is a function Fin k → Fin n, and an agent may receive no task. Types are positive reals, and every truthfulness and approximation quantifier ranges over positive true types, positive misreports and positive declarations of the others.
  • Payments are handed to the agent, so utility is the payment minus the true time spent. Payments are computed from the declared vector, never from true types.
  • The allocation rule is a parameter satisfying the MinWork specification (IsMinWorkAlloc). Every result holds for every tie-breaking rule, including rules that depend on the whole declared vector. No particular argmin is fixed.
  • n≥2n \ge 2n≥2 is a hypothesis of the goal and of the truthfulness items: with a single agent the paper's second-best minimum is undefined. The approximation items need only n≥1n \ge 1n≥1. There is no hypothesis on kkk.
  • The make-span and both minima are Finset.sup' / Finset.inf' over nonempty finite sets, so they are true maxima and minima with no default values.
  • Strong truthfulness is formalized as truthfulness plus: every misreport di≠tid^i \ne t^idi=ti is strictly worse than the truth for some positive declarations of the others. Given truthfulness this is equivalent to Definition 5. A formalization that states only that truth-telling is dominant, or proves strictness only for single-task instances, does not meet the goal. Neither does an existential ratio in place of nnn.
  • Printed slip: in the proof of Claim 4.2 (p. 177) the case di>tid^i > t^idi>ti reads "the utility for agent iii is ti−di<0t^i - d^i < 0ti−di<0, instead of 0 in the case of truth-telling". With the Definition 11 payments the misreporting agent loses the task (utility 0), and the truthful agent wins it with utility d3−i−ti>0d^{3-i} - t^i > 0d3−i−ti>0. The milestone text keeps the paper's words; the Lean statements assert what the argument establishes.
  • Out of scope: running time ("polynomial time"), and the paper's general revelation-principle framework (Proposition 2.1).
  • Welcome contributions: proofs of the milestones, and a reusable lemma connecting the local VGC milestone to AGT.vcg_incentive_compatible.

Selected references

  • N. Nisan, A. Ronen, Algorithmic Mechanism Design, Games and Economic Behavior 35 (2001) 166–196. https://doi.org/10.1006/game.1999.0790
  • T. Groves, Incentives in Teams, Econometrica 41 (1973) 617–631. https://doi.org/10.2307/1914085
  • W. Vickrey, Counterspeculation, Auctions, and Competitive Sealed Tenders, Journal of Finance 16 (1961) 8–37. https://doi.org/10.1111/j.1540-6261.1961.tb02789.x
  • J. K. Lenstra, D. B. Shmoys, É. Tardos, Approximation algorithms for scheduling unrelated parallel machines, Mathematical Programming 46 (1990) 259–271. https://doi.org/10.1007/BF01585745
  • G. Christodoulou, E. Koutsoupias, A. Kovács, A Proof of the Nisan-Ronen Conjecture, STOC 2023. https://arxiv.org/abs/2301.11905
9 thms4 active usersReviewed
PreviousPage 2 of 36Next

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paper
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me