Prove2Me
Navigate
DiscoverFormalpediaBlogsUsersMomentumMy Missions+
Prove2Me
⌕
Log in

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

Operations Research

1,094 missions · 575 completed

The discipline of applying mathematical analysis to complex decision problems in operations: allocating scarce resources, scheduling, routing, inventory, and the design of service and production systems. Drawing on mathematical programming, stochastic modeling, queueing, simulation, and game-theoretic reasoning, it seeks policies that perform provably well in systems shaped by constraints, congestion, and uncertainty.

Missions

Open519Completed575All1094
🏆Completed
CombinatoricsGraph TheoryOptimization+1·Captain: mikedeng1

Applied Combinatorics VII: Minimum Spanning Trees and Dijkstra's AlgorithmTextbook

Motivation

Two optimization problems on weighted networks sit at the base of operations research and algorithm design. The first asks for the cheapest way to connect every node of a network, such as a cable, pipeline or communication network. The answer is a minimum weight spanning tree. The second asks for the shortest route from a depot to every other node of a road or data network, the single-source shortest path problem. Chapter 12 of Keller and Trotter's Applied Combinatorics (appliedcombinatorics.org, CC BY-SA 4.0) treats both. It proves the structural lemmas behind the greedy spanning tree algorithms of Kruskal (1956) and Prim (1957), and the correctness of the shortest path algorithm of Dijkstra (1959).

The minimum spanning tree problem goes back to Borůvka (1926), who designed an electrical network for Moravia. Kruskal and Prim gave the two greedy algorithms taught today, and Dijkstra's 1959 note treated both problems. Dijkstra's shortest path method, with heap-based refinements such as Fredman and Tarjan (1987), remains the standard solver for non-negative lengths and a building block of routing, scheduling and network flow codes.

Setting

A graph G=(V,E)G = (V, E)G=(V,E) has a finite vertex set VVV and a set EEE of 2-element subsets of VVV. A weight w(e)∈N0w(e) \in \mathbb N_0w(e)∈N0​ is attached to each edge, and a set SSS of edges has weight w(S)=∑e∈Sw(e)w(S) = \sum_{e \in S} w(e)w(S)=∑e∈S​w(e). A spanning forest of GGG is an acyclic graph H=(V,S)H = (V, S)H=(V,S) with S⊆ES \subseteq ES⊆E. A spanning tree is a spanning forest that is connected. The weight of a spanning tree is the weight of its edge set. In Lean these are SimpleGraph V with [Fintype V], IsSpanningForest G H (H≤GH \le GH≤G and acyclic), IsSpanningTree G T (T≤GT \le GT≤G and a tree), and weight w T for a weight w : Sym2 V → ℕ.

A digraph G=(V,E)G = (V, E)G=(V,E) has E⊆V×VE \subseteq V \times VE⊆V×V with x≠yx \ne yx=y for every directed edge (x,y)(x, y)(x,y). Each directed edge has a length w(x,y)∈N0w(x, y) \in \mathbb N_0w(x,y)∈N0​. The length is extended by w(x,y)=∞w(x, y) = \inftyw(x,y)=∞ for non-edges. A directed path from aaa to bbb is a sequence (a=u0,…,ut=b)(a = u_0, \dots, u_t = b)(a=u0​,…,ut​=b) of distinct vertices in which consecutive pairs are directed edges. Its length is ∑i<tw(ui,ui+1)\sum_{i<t} w(u_i, u_{i+1})∑i<t​w(ui​,ui+1​). The distance dist⁡(a,b)∈N0∪{∞}\operatorname{dist}(a, b) \in \mathbb N_0 \cup \{\infty\}dist(a,b)∈N0​∪{∞} is the minimum length of a directed path from aaa to bbb, and is ∞\infty∞ when no such path exists. A shortest path is a directed path attaining it. In Lean this is WeightedDigraph V with ext, IsDirPath, pathLength, dist and IsShortestPath.

Dijkstra's algorithm (Algorithm 12.14) with root rrr and n=∣V∣n = |V|n=∣V∣ keeps a sequence σ\sigmaσ of permanent vertices, a value δ(x)∈N0∪{∞}\delta(x) \in \mathbb N_0 \cup \{\infty\}δ(x)∈N0​∪{∞} and a sequence P(x)P(x)P(x) for each vertex. Step 1 sets δ(r)=0\delta(r) = 0δ(r)=0, P(r)=(r)P(r) = (r)P(r)=(r), σ=(r)\sigma = (r)σ=(r), and δ(x)=w(r,x)\delta(x) = w(r, x)δ(x)=w(r,x), P(x)=(r,x)P(x) = (r, x)P(x)=(r,x) for x≠rx \ne rx=r. Step iii with 1<i<n1 < i < n1<i<n scans from the last permanent vertex viv_ivi​. For every temporary xxx it sets δ(x)←min⁡{δ(x),δ(vi)+w(vi,x)}\delta(x) \leftarrow \min\{\delta(x), \delta(v_i) + w(v_i, x)\}δ(x)←min{δ(x),δ(vi​)+w(vi​,x)}, and on a strict decrease it replaces P(x)P(x)P(x) by P(vi)P(v_i)P(vi​) followed by xxx. Each step ends by appending to σ\sigmaσ a temporary vertex of minimum δ\deltaδ, chosen arbitrarily among ties. The algorithm halts at Step nnn. DijkstraRun G r i s holds when some sequence of admissible choices leads to state s at the start of Step iii.

Formalization targets

Goal: correctness of Dijkstra's algorithm (Theorem 12.18)

For every halted state of every run, and every vertex xxx,

δ(x)=dist⁡(r,x),dist⁡(r,x)<∞  ⟹  P(x) is a shortest path from r to x.\delta(x) = \operatorname{dist}(r, x), \qquad \operatorname{dist}(r, x) < \infty \implies P(x) \text{ is a shortest path from } r \text{ to } x.δ(x)=dist(r,x),dist(r,x)<∞⟹P(x) is a shortest path from r to x.

Milestones

  1. Proposition 12.3. A spanning forest H=(V,S)H = (V, S)H=(V,S) of a graph on n≥1n \ge 1n≥1 vertices has ∣S∣≤n−1|S| \le n - 1∣S∣≤n−1 and exactly n−∣S∣n - |S|n−∣S∣ components. It is a spanning tree if and only if ∣S∣=n−1|S| = n - 1∣S∣=n−1.
  2. Proposition 12.4 (Exchange Principle). Let TTT be a spanning tree and xy∈E∖Txy \in E \setminus Txy∈E∖T. Then TTT contains a unique path x=x0,…,xt=yx = x_0, \dots, x_t = yx=x0​,…,xt​=y, and replacing any edge xixi+1x_i x_{i+1}xi​xi+1​ of it by xyxyxy gives a spanning tree.
  3. Lemma 12.6. In a connected weighted graph, let FFF be a spanning forest and CCC a component of FFF. A minimum weight edge leaving CCC lies in some spanning tree that has minimum weight among the spanning trees containing FFF.
  4. Proposition 12.16. Every prefix and every suffix of a shortest path is a shortest path.
  5. Proposition 12.17. When the algorithm halts, δ(v1)≤δ(v2)≤⋯≤δ(vn)\delta(v_1) \le \delta(v_2) \le \cdots \le \delta(v_n)δ(v1​)≤δ(v2​)≤⋯≤δ(vn​).

Milestones 4 and 5 are the two statements the book's proof of the goal rests on. Milestones 1–3 are the spanning tree half of the chapter. Lemma 12.6 is the result from which the book derives the correctness of Kruskal's and Prim's algorithms.

Significance

Theorem 12.18 certifies that one pass of nnn steps computes all distances from rrr and a shortest path tree, with no condition on the digraph beyond non-negative lengths. Lemma 12.6 is the cut property. Every greedy minimum spanning tree method (Kruskal, Prim, Borůvka) is an instance of it, and the exchange principle is the matroid basis-exchange axiom specialised to the graphic matroid.

All of these results are classical and proved. None is formalized in this form on the platform. Mathlib has spanning trees of connected graphs, uniqueness of paths in acyclic graphs, and the edge count n−1n - 1n−1 of a tree. It has no edge–component count for forests, no exchange principle, no weighted spanning trees, and no Dijkstra. On Prove2Me, FamousTheorems.tree_card_edges_6b and ClassicalGaps.isAcyclic_edges_eq_card_sub_one_imp_connected cover only the tree case of Proposition 12.3. KServer.mst_cut_property is a cut property for complete graphs encoded by parent maps, a different statement. The label-correcting algorithm of Dynamic Programming and Optimal Control II (BertsekasDP.label_correcting_*) is a different algorithm: it keeps an open list and scans in arbitrary order, not by minimum label.

Difficulty

The goal is a statement about the final state of a run, but the facts it depends on only become visible across steps: a permanent vertex's δ\deltaδ and PPP never change again, and δ(x)\delta(x)δ(x) is always the length of the current P(x)P(x)P(x). None of this is recorded in the final state itself. An argument over the steps of the run has to show that each P(x)P(x)P(x) remains a path with distinct vertices, including when edges of length 000 allow ties. It also has to handle the value ∞\infty∞, where ∞+a=∞\infty + a = \infty∞+a=∞ and a comparison between two infinite values never counts as a decrease. Tie-breaking is arbitrary, so no argument may depend on which minimum is chosen. For Lemma 12.6 the difficulty is the exchange step: removing an edge of a tree path and adding a crossing edge must again give a tree that still contains the forest FFF, and this is a statement about cycles and components, not about counts.

Formalization scope

  • Graphs are SimpleGraph V over a Fintype V. Weights are Sym2 V → ℕ (the book's w:E→N0w : E \to \mathbb N_0w:E→N0​; values off EEE are never used). Acyclic, tree and connected components are Mathlib's. In Proposition 12.3, ∣S∣=n−k|S| = n - k∣S∣=n−k is written ∣S∣+k=n|S| + k = n∣S∣+k=n and n≥1n \ge 1n≥1 is assumed, which the bound n−1n - 1n−1 presupposes.
  • Lemma 12.6 assumes GGG connected, the section's standing assumption (p. 239). The page's "to avoid trivialities, we assume n≥3n \ge 3n≥3" is not imposed, because the statement holds for every nnn. The crossing edge may have either endpoint in CCC.
  • Lengths in the digraph are ℕ, and δ\deltaδ and distances are ℕ∞, where ∞\infty∞ is ⊤, never a large finite number. A version with real or ℝ≥0 lengths would be a generalization and is not what is asked.
  • Dijkstra's algorithm is defined step by step exactly as on pp. 246–247, including δ(x)=w(r,x)=∞\delta(x) = w(r, x) = \inftyδ(x)=w(r,x)=∞ and P(x)=(r,x)P(x) = (r, x)P(x)=(r,x) for non-neighbours at Step 1. The goal quantifies over every halted state, so it holds for every tie-breaking. A halted state always exists; a sorry-free check of this is in the workspace. For a vertex not reachable from rrr the book is silent. The distance there is read as ∞\infty∞, and the shortest-path conclusion is asserted only at finite distance.
  • A trivializing formalization is ruled out: δ\deltaδ is computed by the update rule of Algorithm 12.14, not defined as the distance, and the theorem is not stated for an arbitrary procedure satisfying its own conclusion.
  • The book uses no O(⋅)O(\cdot)O(⋅) bounds or approximate constants in these statements, so there are no constants to instantiate.
  • Reusable infrastructure: a list-based theory of directed paths and distances in ℕ∞, the invariants of Dijkstra's algorithm, and forest edge counting. Contributions of general lemmas (walks shortcut to paths without increasing length, component counts under edge insertion) are welcome.

Selected references

  • M. T. Keller and W. T. Trotter, Applied Combinatorics, 2017 Edition, Chapter 12. https://www.appliedcombinatorics.org/
  • E. W. Dijkstra, "A note on two problems in connexion with graphs", Numerische Mathematik 1 (1959) 269–271. https://doi.org/10.1007/BF01386390
  • J. B. Kruskal, "On the shortest spanning subtree of a graph and the traveling salesman problem", Proc. AMS 7 (1956) 48–50. https://doi.org/10.1090/S0002-9939-1956-0078686-7
  • R. C. Prim, "Shortest connection networks and some generalizations", Bell System Technical Journal 36 (1957) 1389–1401. https://doi.org/10.1002/j.1538-7305.1957.tb01515.x
  • M. L. Fredman and R. E. Tarjan, "Fibonacci heaps and their uses in improved network optimization algorithms", J. ACM 34 (1987) 596–615. https://doi.org/10.1145/28869.28874
  • O. Borůvka, "O jistém problému minimálním", Práce Moravské přírodovědecké společnosti 3 (1926) 37–58. https://dml.cz/handle/10338.dmlcz/500114
8 thms2 active usersReviewed
Convex OptimizationDiscrete GeometryOptimization·Captain: Shuze Chen

Discrete Convex Analysis XXVIII: The Conjugacy TheoremTextbook

Motivation

Chapter 8 is where discrete convex analysis explains why it needed two separate notions — M-convexity (exchangeability) and L-convexity (submodularity) — rather than one. The answer is conjugacy: under the classical Legendre-Fenchel transform, the two classes turn out to be exactly dual to each other, the discrete analogue of the fact that convex analysis's transform is self-dual within a single class of convex functions. Mission 10-conjugacy-i proved the integer-lattice version of this fact (Theorem 8.12) but explicitly deferred the polyhedral version — Theorem 8.4, the chapter's own headline "Conjugacy theorem" — noting it needed a real-variable M-/L-convex-function layer the series had not yet built. That layer now exists, built across missions 23-24-ch06*-mconvexfunctions and 26-27-ch07*-lconvexfunctions. This mission proves Theorem 8.4 and its companions: the polar-cone correspondence it induces, its nonpolyhedral generalization, the separation and Fenchel-duality theorems for M♮-/L♮-convex functions, and the basic theory of M2-convex functions (sums of M-convex functions), which the Edmonds intersection theorem's own combinatorics is built from.

Setting

Fix a finite ground set VVV. For f:RV→R∪{+∞}f : \mathbb R^V \to \mathbb R \cup \{+\infty\}f:RV→R∪{+∞}, the Legendre-Fenchel transform is f∙(p)=sup⁡x[⟨p,x⟩−f(x)]f^\bullet(p) = \sup_x [\langle p,x\rangle - f(x)]f∙(p)=supx​[⟨p,x⟩−f(x)]. A polyhedral convex function fff is M-convex (f∈M[R→R]f \in M[\mathbb R \to \mathbb R]f∈M[R→R]) if it satisfies (M-EXC[R]); ggg is L-convex (g∈L[R→R]g \in L[\mathbb R \to \mathbb R]g∈L[R→R]) if it satisfies (SBF[R]) and (TRF[R]). A concave function hhh is always represented via h2=−hh_2 = -hh2​=−h, an ordinary convex function, so every "f≥hf \ge hf≥h" hypothesis is restated as "f+h2≥0f + h_2 \ge 0f+h2​≥0" — an equivalent formulation avoiding any need to represent −∞-\infty−∞ in the codomain. A polyhedral cone's polar is C∘={y:⟨y,x⟩≤0 ∀x∈C}C^\circ = \{y : \langle y,x\rangle \le 0\ \forall x \in C\}C∘={y:⟨y,x⟩≤0 ∀x∈C}. A function is M2-convex if it is the sum of two M-convex functions.

Formalization targets

Goal: the conjugacy theorem (Theorem 8.4)

The classes of polyhedral M-convex functions and polyhedral L-convex functions are in one-to-one correspondence under the Legendre-Fenchel transform: f∈M⇒f∙∈Lf \in M \Rightarrow f^\bullet \in Lf∈M⇒f∙∈L, g∈L⇒g∙∈Mg \in L \Rightarrow g^\bullet \in Mg∈L⇒g∙∈M, and the transform is an involution (f∙∙=ff^{\bullet\bullet}=ff∙∙=f, g∙∙=gg^{\bullet\bullet}=gg∙∙=g) on each class, with the identical statement for the M♮^\natural♮/L♮^\natural♮ variants. This is the theorem mission 10-conjugacy-i deferred, citing exactly the missing infrastructure this series has since built.

Supporting structural targets

Twelve further results build the surrounding theory. Proposition 8.2 gives the easy two-variable case of the general submodularity-preservation fact (Theorem 8.1, already a milestone of mission 10-conjugacy-i); Proposition 8.3 is the technical minimizer-difference lemma the goal's harder direction is built from. Theorem 8.5 derives the M-convex/L-convex cone polarity from the goal, and Theorem 8.6 extends the correspondence beyond the polyhedral case to general closed proper convex functions. Proposition 8.14 and Theorems 8.15-8.16 build the separation theory for M♮-/L♮-convex and concave function pairs, with integral witnesses when the functions are integer valued; Theorem 8.21 (parts 1-2) derives the Fenchel-type strong-duality equality these separation theorems make possible. Propositions 8.29-8.30 and Theorem 8.31 (plus Theorem 8.32, found by direct reading immediately after 8.31) build the basic theory of M2-convex functions: their domains and minimizer sets are M2-convex, they are integrally convex, and their global optimality reduces to a finite local check.

Significance

The goal is the theorem that retroactively explains this entire series' two-track structure: missions 20-25 (M-convex sets and functions) and 08/21/26-28 (L-convex sets and functions) are not two independent theories that happen to share techniques — they are conjugate images of each other, so every theorem proved on one side has a dual counterpart automatically available on the other via Theorem 8.4. This is made concrete immediately: Theorem 8.5's cone polarity and the diagram the book draws connecting M0[R]M_0[\mathbb R]M0​[R], 0L[R→R]0L[\mathbb R\to\mathbb R]0L[R→R], and submodular set functions S[R]S[\mathbb R]S[R] (already correspondences this series proved independently, in missions 24-ch06d-mconvexfunctions and 28-ch07d-lconvexfunctions) are shown to be facets of one single conjugacy fact rather than three separate coincidences. The separation and Fenchel duality theorems (8.15, 8.16, 8.21) are the discrete analogues of the two theorems every convex optimization course opens with, and the book is explicit that they are not corollaries of the classical versions plus convex extensibility — they carry genuinely combinatorial content, specializing to Frank's discrete separation theorem and Edmonds's intersection theorem as examples the book itself gives.

None of these results are open — they are Murota's account of the duality at the heart of discrete convex analysis, the reason the theory needed two dual notions rather than one. What this mission contributes is a faithful, machine-checked formal statement of each, completing a theorem mission 10-conjugacy-i explicitly left for a future session once the necessary polyhedral apparatus existed, and including one result (Theorem 8.32) the platform's own automated extractor missed; no comparable formalization exists on the platform (see Formalization scope).

Difficulty

The naive approach to the goal's harder direction (L⇒M) would try to verify the exchange inequality for g∙g^\bulletg∙ directly from the definition of the transform; the book's actual proof instead identifies the exchange inequality with a statement about weighted minimizers of ggg itself via Proposition 8.3 (the minimizer-difference bound), converting a claim about the conjugate function into a claim about ggg's own combinatorial structure — a genuine change of perspective, not a direct calculation. Proposition 8.3's own proof is the hardest single argument in this block: it derives the minimizer-difference bound by a contradiction argument that constructs an explicit pair of "worse" minimizers via a join/meet perturbation and derives a strict inequality from Theorem 7.29's translation inequality — a multi-step combinatorial argument with no direct shortcut.

Formalization scope

Ground-set elements are a Fintype V with DecidableEq; convex functions are WithTop ℝ valued throughout (never EReal, except for the Legendre-Fenchel transform itself, whose defining supremum/infimum can genuinely be infinite). All thirteen numbered results found in this chunk's page range — the twelve in BRIEF.md's own table plus Theorem 8.32 — are placed, with one documented scope reduction: Theorem 8.21 states only its real-attainment parts (1)-(2), not the integer-attainment refinement of parts (3)-(4), which needs a separate argument no other result in this chunk requires — see HARD.md. Concave functions hhh are always represented via h2=−hh_2 = -hh2​=−h and every inequality f≥hf \ge hf≥h restated as f+h2≥0f + h_2 \ge 0f+h2​≥0, avoiding WithTop ℝ negation entirely. This chunk's own BRIEF.md inherited the chapters-4-7 page-offset boilerplate (printed = PDF −-− 19); chapter 8 uses offset 18, confirmed against the PDF's own footers — every citation here uses the corrected offset. This mission's base vocabulary is redeclared from missions 10-conjugacy-i, 20-ch04b-mconvexsets, 21-ch05b-lconvexsets, 23-24-ch06*-mconvexfunctions, and 26-27-ch07*-lconvexfunctions rather than imported, since sibling drafts in this series cannot yet reference one another. Contributions completing any of the thirteen sorrys are welcome; the goal and Proposition 8.3 carry the most independent proof content.

Selected references

  • K. Murota, Discrete Convex Analysis, SIAM, 2003. DOI: 10.1137/1.9780898718508.
  • K. Murota and A. Shioura, "M-convex function on generalized polymatroid," Mathematics of Operations Research, 24 (1999), pp. 95-105 [152] (the polyhedral M-/L-convex conjugacy theory this mission's real-variable results are drawn from).
73 thms2 active usersReviewed
🏆Completed
Linear OptimizationOptimization·Captain: mikedeng1

Revenue Management Under the Markov Chain Choice Model III: The Reduced Linear Program Is Equivalent to the Choice-Based Linear ProgramResearch Paper

Motivation

Network revenue management decides which products to make available to arriving customers when products share scarce resources: an airline sells itineraries (products) that consume seats on flight legs (resources), and each itinerary has a fare. When customers choose among the offered products, rather than asking for one fixed product, the standard planning tool is a deterministic linear program that replaces random choices by their expected values. Gallego, Iyengar, Phillips and Dubey (2004, Columbia CORC technical report TR-2004-01) and Liu and van Ryzin (2008) formulated this choice-based linear program; its solution drives bid-price and offer-set policies used in practice.

The difficulty is size. The choice-based program has one variable for each subset of products, 2n2^n2n in all, and is solved by column generation, whose pricing subproblem is itself an assortment problem. Feldman and Topaloglu (Oper. Res. 65(5), 2017) show that when customers choose under the Markov chain choice model of Blanchet, Gallego and Goyal (2016), the choice-based program is equivalent to a linear program with only 2n2n2n variables and m+nm+nm+n constraints. This mission formalizes that equivalence, Theorem 7 of the paper.

Setting

There are nnn products, N={1,…,n}N=\{1,\dots,n\}N={1,…,n}. Under the Markov chain choice model a customer first visits product jjj with probability λj\lambda_jλj​. If the product she visits is offered, she buys it. Otherwise she moves from product jjj to product iii with probability ρj,i\rho_{j,i}ρj,i​, or leaves without buying with probability 1−∑i∈Nρj,i1-\sum_{i\in N}\rho_{j,i}1−∑i∈N​ρj,i​. The paper assumes throughout that

λj>0and∑i∈Nρj,i<1for all j∈N.\lambda_j>0\quad\text{and}\quad\sum_{i\in N}\rho_{j,i}<1\qquad\text{for all } j\in N.λj​>0andi∈N∑​ρj,i​<1for all j∈N.

For an offer set S⊆NS\subseteq NS⊆N, Pj,SP_{j,S}Pj,S​ is the expected number of visits to product jjj while it is offered, which is its purchase probability, and Rj,SR_{j,S}Rj,S​ is the expected number of visits to jjj while it is not offered. The pair (PS,RS)(P_S,R_S)(PS​,RS​) is the solution of the (Balance) equations

Pj,S+Rj,S=λj+∑i∈Nρi,jRi,S  ∀j∈N,Pj,S=0  ∀j∉S,Rj,S=0  ∀j∈S.P_{j,S}+R_{j,S}=\lambda_j+\sum_{i\in N}\rho_{i,j}R_{i,S}\ \ \forall j\in N,\qquad P_{j,S}=0\ \ \forall j\notin S,\qquad R_{j,S}=0\ \ \forall j\in S .Pj,S​+Rj,S​=λj​+i∈N∑​ρi,j​Ri,S​  ∀j∈N,Pj,S​=0  ∀j∈/S,Rj,S​=0  ∀j∈S.

Dropping the constraints tied to SSS gives the polyhedron

H={(x,z)∈R+2n:xj+zj=λj+∑i∈Nρi,jzi  ∀j∈N}.\mathcal H=\Big\{(x,z)\in\mathbb R^{2n}_+ : x_j+z_j=\lambda_j+\sum_{i\in N}\rho_{i,j}z_i\ \ \forall j\in N\Big\}.H={(x,z)∈R+2n​:xj​+zj​=λj​+i∈N∑​ρi,j​zi​  ∀j∈N}.

The network has mmm resources, M={1,…,m}M=\{1,\dots,m\}M={1,…,m}, with capacities cqc_qcq​; the selling horizon has TTT periods; product jjj earns rjr_jrj​ and consumes aq,ja_{q,j}aq,j​ units of resource qqq. With uSu_SuS​ the probability of offering SSS in a period, the (Choice Based) linear program is

max⁡u∈R+2n{∑S⊆N∑j∈NTrjPj,SuS: ∑S⊆N∑j∈NTaq,jPj,SuS≤cq ∀q∈M, ∑S⊆NuS=1},\max_{u\in\mathbb R^{2^n}_+}\Big\{\sum_{S\subseteq N}\sum_{j\in N}T r_jP_{j,S}u_S:\ \sum_{S\subseteq N}\sum_{j\in N}Ta_{q,j}P_{j,S}u_S\le c_q\ \forall q\in M,\ \sum_{S\subseteq N}u_S=1\Big\},u∈R+2n​max​{S⊆N∑​j∈N∑​Trj​Pj,S​uS​: S⊆N∑​j∈N∑​Taq,j​Pj,S​uS​≤cq​ ∀q∈M, S⊆N∑​uS​=1},

and the (Reduced) linear program is

max⁡(x,z)∈R+2n{∑j∈NTrjxj: ∑j∈NTaq,jxj≤cq ∀q∈M, xj+zj=λj+∑i∈Nρi,jzi ∀j∈N}.\max_{(x,z)\in\mathbb R^{2n}_+}\Big\{\sum_{j\in N}T r_jx_j:\ \sum_{j\in N}Ta_{q,j}x_j\le c_q\ \forall q\in M,\ x_j+z_j=\lambda_j+\sum_{i\in N}\rho_{i,j}z_i\ \forall j\in N\Big\}.(x,z)∈R+2n​max​{j∈N∑​Trj​xj​: j∈N∑​Taq,j​xj​≤cq​ ∀q∈M, xj​+zj​=λj​+i∈N∑​ρi,j​zi​ ∀j∈N}.

In (Reduced), xjx_jxj​ is the expected number of visits to product jjj while it is available and zjz_jzj​ the expected number of visits while it is not.

Formalization targets

Goal: Theorem 7

Let (x^,z^)(\hat x,\hat z)(x^,z^) be an optimal solution of (Reduced). Then there are subsets S1,…,SK⊆NS^1,\dots,S^K\subseteq NS1,…,SK⊆N and positive scalars γ1,…,γK\gamma^1,\dots,\gamma^Kγ1,…,γK with ∑kγk=1\sum_k\gamma^k=1∑k​γk=1 such that

x^=∑k=1KγkPSk,z^=∑k=1KγkRSk,\hat x=\sum_{k=1}^K\gamma^kP_{S^k},\qquad \hat z=\sum_{k=1}^K\gamma^kR_{S^k},x^=k=1∑K​γkPSk​,z^=k=1∑K​γkRSk​,

and for any such subsets and scalars the vector u^\hat uu^ with u^Sk=γk\hat u_{S^k}=\gamma^ku^Sk​=γk and u^S=0\hat u_S=0u^S​=0 for S∉{S1,…,SK}S\notin\{S^1,\dots,S^K\}S∈/{S1,…,SK} is optimal for (Choice Based), with objective value equal to that of (x^,z^)(\hat x,\hat z)(x^,z^) in (Reduced). In particular the two programs have the same optimal value.

Milestones

  1. (Balance) has a unique and nonnegative solution for every offer set (§2, p. 1325).
  2. Lemma 1: an extreme point (x^,z^)(\hat x,\hat z)(x^,z^) of H\mathcal HH equals (PS,RS)(P_{S},R_{S})(PS​,RS​) for S={j:x^j>0}S=\{j:\hat x_j>0\}S={j:x^j​>0} (p. 1326).
  3. Lemma 10: H\mathcal HH is bounded (quoted on p. 1331; proved in the online appendix).
  4. Every point of H\mathcal HH is a positive convex combination of finitely many extreme points of H\mathcal HH (proof of Theorem 7, p. 1331).
  5. Every feasible uuu of (Choice Based) yields the feasible point x~j=∑SPj,SuS\tilde x_j=\sum_S P_{j,S}u_Sx~j​=∑S​Pj,S​uS​, z~j=∑SRj,SuS\tilde z_j=\sum_S R_{j,S}u_Sz~j​=∑S​Rj,S​uS​ of (Reduced), with the same objective value (proof of Theorem 7, p. 1332).

Significance

The result. Theorem 7 replaces a program with 2n2^n2n columns by one with 2n2n2n variables and m+nm+nm+n constraints, solvable directly by any LP solver, and it returns an optimal solution of the original program, not only its value. The optimal value is the standard upper bound on the optimal expected revenue of a network revenue management policy, and the dual variables of the capacity constraints are the bid prices used to control sales. The decomposition of part 1 is what turns the small program's solution back into offer-set frequencies that a policy can implement; Section 7 of the paper makes that decomposition algorithmic (a separate mission of this series).

Formalizing it. The theorem is proved in the paper; to the best of available knowledge no machine-checked version exists. A formal proof checks the link between polyhedral geometry (extreme points of H\mathcal HH and the solutions of (Balance)) and linear-programming optimality, and records exactly which properties of the Markov chain choice model are used: the standing assumptions enter through uniqueness and nonnegativity of (PS,RS)(P_S,R_S)(PS​,RS​) and through boundedness of H\mathcal HH.

Difficulty

The inequality "(Reduced) ≥\ge≥ (Choice Based)" is a direct computation: averaging the (Balance) equations with weights uSu_SuS​ lands in H\mathcal HH. The reverse direction is where the obvious argument fails. A point of H\mathcal HH has no offer set attached to it, and a general polyhedron need not be the convex hull of its extreme points: it can contain lines or rays. The argument requires that H\mathcal HH is bounded, which depends on the substochasticity ∑iρj,i<1\sum_i\rho_{j,i}<1∑i​ρj,i​<1, and that each extreme point is exactly some (PS,RS)(P_S,R_S)(PS​,RS​), which uses the structure of the balance equations and the uniqueness of their solution. Neither follows from general linear-programming facts. In Lean, the finite vertex representation of a bounded polyhedron is also not a one-line consequence of Mathlib's Krein–Milman theorem, which gives only the closure of the convex hull.

Formalization scope

Products are Fin n, offer sets Finset (Fin n), resources Fin m. The model is a structure Model n holding λ\lambdaλ, ρ\rhoρ (rho j i =ρj,i=\rho_{j,i}=ρj,i​, the transition from jjj to iii) and the standing assumptions λj>0\lambda_j>0λj​>0 and ∑iρj,i<1\sum_i\rho_{j,i}<1∑i​ρj,i​<1; the nonnegativity ρj,i≥0\rho_{j,i}\ge0ρj,i​≥0, implicit in the paper because the ρj,i\rho_{j,i}ρj,i​ are probabilities, is an added field. (PS,RS)(P_S,R_S)(PS​,RS​) is a solution of (Balance) chosen by Classical.epsilon, and milestone 1 is what identifies it with the paper's unique solution. H\mathcal HH is a subset of (Fin n → ℝ) × (Fin n → ℝ), extreme points are Mathlib's Set.extremePoints ℝ, and boundedness is Bornology.IsBounded. TTT is a natural number entering as a real factor exactly where the paper writes it; ccc, aaa, rrr carry no sign conditions, as in the paper. "Optimal solution" means feasible and at least as good as every feasible point; no supremum is used.

Two conventions are disclosed. The paper defines u^\hat uu^ by u^Sk=γk\hat u_{S^k}=\gamma^ku^Sk​=γk; the formalization sets u^S=∑k:Sk=Sγk\hat u_S=\sum_{k:S^k=S}\gamma^ku^S​=∑k:Sk=S​γk, which agrees when the SkS^kSk are distinct and is the only consistent reading otherwise. Milestone 5 is stated for every feasible uuu of (Choice Based), while the paper applies it to an optimal one; its argument uses only feasibility. No printed statement needed correction.

The goal is not only the decomposition of part 1, which is milestones 2 and 4 combined: it also asserts optimality of u^\hat uu^ and equality of the optimal values, and a formalization that drops part 2 does not state Theorem 7. Part 2 is required for every decomposition, and part 1 guarantees one exists, so part 2 is not vacuous.

A complete development needs the vertex representation of polytopes (reusable beyond this mission; LinearOptimization.polyhedron_resolution on the platform proves the resolution theorem in another encoding), the theory of substochastic matrices behind (Balance) (invertibility of I−QˉI-\bar QI−Qˉ​ with a nonnegative inverse), and finite-sum manipulations over Finset (Fin n). Contributions to any milestone, and bridges to existing polyhedral results, are welcome.

Selected references

  • J. B. Feldman, H. Topaloglu, Revenue Management Under the Markov Chain Choice Model, Operations Research 65(5):1322–1342, 2017. https://doi.org/10.1287/opre.2017.1628
  • J. Blanchet, G. Gallego, V. Goyal, A Markov Chain Approximation to Choice Modeling, Operations Research 64(4):886–905, 2016. https://doi.org/10.1287/opre.2016.1505
  • Q. Liu, G. van Ryzin, On the Choice-Based Linear Programming Model for Network Revenue Management, Manufacturing & Service Operations Management 10(2):288–310, 2008. https://doi.org/10.1287/msom.1070.0172
  • G. Gallego, G. Iyengar, R. Phillips, A. Dubey, Managing Flexible Products on a Network, Computational Optimization Research Center Technical Report TR-2004-01, Columbia University, 2004 (technical report; no DOI).
  • M. L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming, Wiley, 1994. https://doi.org/10.1002/9780470316887
8 thms2 active usersReviewed
🏆Completed
Dynamic ProgrammingOptimization·Captain: mikedeng1

Revenue Management Under the Markov Chain Choice Model II: Optimal Offer Sets Grow with Remaining Capacity and Remaining TimeResearch Paper

Motivation

Airlines, hotels and rental firms sell a fixed stock of capacity (seats on a flight leg, rooms on a night) over a finite selling horizon, and control revenue mainly by deciding which products (fare classes, rate plans) to make available at each moment. When customers substitute between products, the decision in each period is an assortment: a subset of products to offer. Talluri and van Ryzin (Management Science 2004) formulated this single resource revenue management problem as a dynamic program for a general choice model, and showed that structural properties of the optimal policy depend strongly on how customers choose.

The Markov chain choice model of Blanchet, Gallego and Goyal (technical report 2013; Oper. Res. 2016) describes substitution by a Markov chain on the products and approximates a broad class of random-utility models. Feldman and Topaloglu (Oper. Res. 65(5), 2017) study assortment and revenue management problems under this model. This mission formalizes their structural result for the single resource problem (Theorem 5): there is an optimal policy whose offer sets are nested in the remaining capacity and in the remaining time, so that it can be implemented by protection levels, one capacity threshold per product and period.

Setting

There are nnn products N={1,…,n}N=\{1,\dots,n\}N={1,…,n}. A customer arrives to purchase product jjj with probability λj\lambda_jλj​. If jjj is offered she buys it; otherwise she moves to product iii with probability ρj,i\rho_{j,i}ρj,i​, and leaves with probability 1−∑i∈Nρj,i1-\sum_{i\in N}\rho_{j,i}1−∑i∈N​ρj,i​. The paper assumes λj>0\lambda_j>0λj​>0 and ∑i∈Nρj,i<1\sum_{i\in N}\rho_{j,i}<1∑i∈N​ρj,i​<1 for all jjj. For an offer set S⊆NS\subseteq NS⊆N, the purchase probabilities Pj,SP_{j,S}Pj,S​ and the visit counts Rj,SR_{j,S}Rj,S​ of unavailable products solve the (Balance) equations

Pj,S+Rj,S=λj+∑i∈Nρi,jRi,S  ∀j,Pj,S=0  (j∉S),Rj,S=0  (j∈S).P_{j,S}+R_{j,S}=\lambda_j+\sum_{i\in N}\rho_{i,j}R_{i,S}\ \ \forall j,\qquad P_{j,S}=0\ \ (j\notin S),\qquad R_{j,S}=0\ \ (j\in S).Pj,S​+Rj,S​=λj​+i∈N∑​ρi,j​Ri,S​  ∀j,Pj,S​=0  (j∈/S),Rj,S​=0  (j∈S).

With product revenues rjr_jrj​, the (Assortment) problem is max⁡S⊆N∑jPj,Srj\max_{S\subseteq N}\sum_{j}P_{j,S}r_jmaxS⊆N​∑j​Pj,S​rj​, and its (Dual) linear program is

min⁡v∈Rn{∑jλjvj:vj≥rj, vj≥∑iρj,ivi  ∀j}.\min_{v\in\mathbb R^n}\Big\{\sum_{j}\lambda_jv_j : v_j\ge r_j,\ v_j\ge\sum_{i}\rho_{j,i}v_i\ \ \forall j\Big\}.v∈Rnmin​{j∑​λj​vj​:vj​≥rj​, vj​≥i∑​ρj,i​vi​  ∀j}.

In the single resource problem there are TTT periods and ccc units of capacity. In each period at most one customer arrives; a sale of product jjj earns rjr_jrj​ and uses one unit. The optimal expected revenue Vt(x)V_t(x)Vt​(x) from period ttt on with xxx units left satisfies the (Single Resource) dynamic program

Vt(x)=max⁡S⊆N{∑j∈NPj,S{rj+Vt+1(x−1)−Vt+1(x)}}+Vt+1(x),V_t(x)=\max_{S\subseteq N}\Big\{\sum_{j\in N}P_{j,S}\{r_j+V_{t+1}(x-1)-V_{t+1}(x)\}\Big\}+V_{t+1}(x),Vt​(x)=S⊆Nmax​{j∈N∑​Pj,S​{rj​+Vt+1​(x−1)−Vt+1​(x)}}+Vt+1​(x),

with VT+1(x)=0V_{T+1}(x)=0VT+1​(x)=0 and Vt(0)=0V_t(0)=0Vt​(0)=0. An optimal subset S^t(x)\hat S_t(x)S^t​(x) is a maximizer of the problem on the right side. The marginal value of capacity is ΔVt(x)=Vt(x)−Vt(x−1)\Delta V_t(x)=V_t(x)-V_t(x-1)ΔVt​(x)=Vt​(x)−Vt​(x−1).

Formalization targets

Goal: Theorem 5 (p. 1329)

There exists an optimal policy, i.e. a choice of maximizers S^t(x)\hat S_t(x)S^t​(x) for all 1≤t≤T1\le t\le T1≤t≤T, 1≤x≤c1\le x\le c1≤x≤c, such that

S^t(x−1)⊆S^t(x)andS^t−1(x)⊆S^t(x).\hat S_t(x-1)\subseteq \hat S_t(x)\qquad\text{and}\qquad \hat S_{t-1}(x)\subseteq\hat S_t(x).S^t​(x−1)⊆S^t​(x)andS^t−1​(x)⊆S^t​(x).

The offer set shrinks as capacity runs down, and it is smaller when more periods remain.

Milestones

  1. (§2, p. 1325) The (Balance) equations have a unique nonnegative solution for every SSS.
  2. (Theorem 2, p. 1326) If v^\hat vv^ is optimal for (Dual), then {j:v^j=rj}\{j:\hat v_j=r_j\}{j:v^j​=rj​} is optimal for (Assortment).
  3. (Lemma 3, p. 1327) For η≥0\eta\ge0η≥0, with v^η\hat v^\etav^η optimal for (Dual) with revenues rj−ηr_j-\etarj​−η,
{j:v^jη=rj−η}⊆{j:v^j0=rj}.\{j:\hat v^\eta_j=r_j-\eta\}\subseteq\{j:\hat v^0_j=r_j\}.{j:v^jη​=rj​−η}⊆{j:v^j0​=rj​}.
  1. ((Single Resource), pp. 1328–1329) The value functions satisfy the printed recursion Vt(x)=max⁡S{∑jPj,S{rj+Vt+1(x−1)}+{1−∑jPj,S}Vt+1(x)}V_t(x)=\max_S\{\sum_jP_{j,S}\{r_j+V_{t+1}(x-1)\}+\{1-\sum_jP_{j,S}\}V_{t+1}(x)\}Vt​(x)=maxS​{∑j​Pj,S​{rj​+Vt+1​(x−1)}+{1−∑j​Pj,S​}Vt+1​(x)} and the boundary conditions.
  2. (Proof of Theorem 5, p. 1329) ΔVt+1(x)≤ΔVt+1(x−1)\Delta V_{t+1}(x)\le\Delta V_{t+1}(x-1)ΔVt+1​(x)≤ΔVt+1​(x−1) and ΔVt+1(x)≤ΔVt(x)\Delta V_{t+1}(x)\le\Delta V_t(x)ΔVt+1​(x)≤ΔVt​(x).

Significance

Theorem 5 makes the optimal policy a protection level policy: for each product jjj and period ttt there is a threshold xˉjt\bar x_{jt}xˉjt​ such that jjj is offered exactly when at least xˉjt\bar x_{jt}xˉjt​ units remain. The policy can then be stored as n Tn\,TnT numbers instead of a table of subsets, and each product can be controlled separately, which is how airline inventory systems are organized. The paper also shows (Table 2, p. 1330) that the products need not be closed in revenue order: a higher-fare product can be closed before a lower-fare one, so the result is genuinely about nested sets, not about nested fare classes. Talluri and van Ryzin (2004) show that under the multinomial logit model an optimal assortment consists of a number of products with the largest revenues; the Markov chain choice model does not have this property (Table 1, p. 1328), so their structure of the optimal policy does not carry over directly.

The result is proved in the paper. No part of it is formalized on Prove2Me. The monotonicity of marginal values for an abstract choice model is published and proved as RevenueManagement.choice_marginal_values, stated over the definitions RevenueManagement_singleResource; milestone 5 is the same statement for this paper's dynamic program and can be bridged to it. A complete development here produces machine-checked Theorem 2 and Lemma 3 for the Markov chain choice model, which are reusable for any assortment problem under this model.

Difficulty

The obvious argument chooses, for each (t,x)(t,x)(t,x), any maximizer of the stage problem. This fails: the stage problems have ties, and an arbitrary choice of maximizers need not be nested, so the theorem is an existence statement about a coordinated choice. The dependence of Pj,SP_{j,S}Pj,S​ on SSS is through the solution of a linear system, so the effect of adding or removing a product on the other purchase probabilities has no simple sign, and optimal assortments need not be nested by revenue (Table 1, p. 1328). The link between two stage problems is that their revenues differ by the same constant for all products, and what has to be shown is that such a uniform shift moves an optimal assortment in a controlled direction. Revenues in the stage problems, rj−ΔVt+1(x)r_j-\Delta V_{t+1}(x)rj​−ΔVt+1​(x), can be negative.

Formalization scope

Products are Fin n, offer sets Finset (Fin n), the paper's ⊂\subset⊂ is non-strict inclusion ⊆\subseteq⊆. The model is a structure with fields λ\lambdaλ, ρ\rhoρ and the standing assumptions λj>0\lambda_j>0λj​>0, ∑iρj,i<1\sum_i\rho_{j,i}<1∑i​ρj,i​<1; ρj,i≥0\rho_{j,i}\ge0ρj,i​≥0 is added as a field because the ρj,i\rho_{j,i}ρj,i​ are probabilities. (PS,RS)(P_S,R_S)(PS​,RS​) is a solution of (Balance) chosen by Classical.epsilon; milestone 1 states that it is the unique nonnegative one. The value function is defined by recursion on the number of periods to go, and VtV_tVt​ is used only for 1≤t≤T+11\le t\le T+11≤t≤T+1. Maxima over S⊆NS\subseteq NS⊆N are taken over the finite family of all subsets, so they are attained; dual optimality is "feasible and no worse than every feasible point".

One hypothesis is added to Theorem 5 and milestone 5, and disclosed: ∑jλj≤1\sum_j\lambda_j\le1∑j​λj​≤1, implied by the paper's description of at most one arrival per period (it makes 1−∑jPj,S1-\sum_jP_{j,S}1−∑j​Pj,S​ a probability). No sign condition is placed on the revenues, as on the page. Theorem 2 and Lemma 3 are stated for arbitrary real revenues, as the proof of Theorem 5 applies them to rj−ΔVt+1(x)r_j-\Delta V_{t+1}(x)rj​−ΔVt+1​(x). The first inclusion of Theorem 5 is stated for x≥2x\ge2x≥2: with x−1=0x-1=0x−1=0 units there is no decision, and the paper reads S^t(0)\hat S_t(0)S^t​(0) as ∅\emptyset∅. The existential requires every S^t(x)\hat S_t(x)S^t​(x) in range to be optimal for its stage problem; a statement without that clause would be satisfied by the empty sets and is ruled out.

Theorem 2 and Lemma 3 need LP duality for (Dual) and the (Balance) system; milestone 5 needs the standard induction on the dynamic program, or a bridge to the published choice_marginal_values (with arrival probabilities 111 and choice model Pj,SP_{j,S}Pj,S​). Proofs of any milestone, and bridges to published Mathlib or platform LP duality results, are welcome.

Selected references

  • J. B. Feldman, H. Topaloglu, Revenue Management Under the Markov Chain Choice Model, Operations Research 65(5):1322–1342, 2017. https://doi.org/10.1287/opre.2017.1628
  • J. Blanchet, G. Gallego, V. Goyal, A Markov Chain Approximation to Choice Modeling, Operations Research 64(4):886–905, 2016. https://doi.org/10.1287/opre.2016.1505
  • K. Talluri, G. van Ryzin, Revenue Management Under a General Discrete Choice Model of Consumer Behavior, Management Science 50(1):15–33, 2004. https://doi.org/10.1287/mnsc.1030.0147
  • K. Talluri, G. van Ryzin, The Theory and Practice of Revenue Management, Springer, 2004. https://doi.org/10.1007/b139000
10 thms2 active usersReviewed
🏆Completed
Linear OptimizationOptimization·Captain: mikedeng1

Revenue Management Under the Markov Chain Choice Model I: The Dual Linear Program Yields an Optimal AssortmentResearch Paper

Motivation

A retailer or an airline decides which products to make available, and customers choose among what is offered. When a preferred product is missing, many customers substitute to another product instead of leaving. Assortment optimization asks which subset of products to offer so that the expected revenue from a customer is as large as possible, and its answer depends entirely on the choice model used to describe substitution.

The Markov chain choice model was introduced by Blanchet, Gallego and Goyal (EC 2013; Oper. Res. 64(4), 2016), who showed that it contains the multinomial logit model as a special case and proposed it as an approximation of general random-utility choice models. Feldman and Topaloglu (Oper. Res. 65(5), 2017) study revenue management under this model. Their first result, the subject of this mission, is that the assortment problem, a search over all 2n2^n2n offer sets, is solved by one linear program with nnn variables. The same paper uses this result for its dynamic single-resource and network results, which are the subjects of the companion missions II–IV of this series.

Timeline:

  • 2013/2016: Blanchet, Gallego and Goyal introduce the model and give a polynomial-time assortment algorithm.
  • 2017: Feldman and Topaloglu show that the optimal assortment is read off an optimal solution of a dual linear program (their Theorem 2), and derive structural and capacity-control consequences.

Setting

There are nnn products N={1,…,n}N=\{1,\dots,n\}N={1,…,n}. A customer arrives to purchase product jjj with probability λj\lambda_jλj​. If the product she visits is offered, she buys it. Otherwise she transitions to product iii with probability ρj,i\rho_{j,i}ρj,i​ and checks whether iii is offered, or leaves without buying with probability 1−∑i∈Nρj,i1-\sum_{i\in N}\rho_{j,i}1−∑i∈N​ρj,i​. Throughout, λj>0\lambda_j>0λj​>0, ρj,i≥0\rho_{j,i}\ge0ρj,i​≥0 and ∑i∈Nρj,i<1\sum_{i\in N}\rho_{j,i}<1∑i∈N​ρj,i​<1 for all jjj.

For an offer set S⊆NS\subseteq NS⊆N, let Pj,SP_{j,S}Pj,S​ be the expected number of visits to product jjj while it is offered (the probability that jjj is purchased) and Rj,SR_{j,S}Rj,S​ the expected number of visits to jjj while it is not offered. The pair (PS,RS)(P_S,R_S)(PS​,RS​) is the unique solution of the (Balance) equations

Pj,S+Rj,S=λj+∑i∈Nρi,jRi,S  ∀j∈N,Pj,S=0  ∀j∉S,Rj,S=0  ∀j∈S.P_{j,S}+R_{j,S}=\lambda_j+\sum_{i\in N}\rho_{i,j}R_{i,S}\ \ \forall j\in N,\qquad P_{j,S}=0\ \ \forall j\notin S,\qquad R_{j,S}=0\ \ \forall j\in S .Pj,S​+Rj,S​=λj​+i∈N∑​ρi,j​Ri,S​  ∀j∈N,Pj,S​=0  ∀j∈/S,Rj,S​=0  ∀j∈S.

With revenue rj∈Rr_j\in\mathbb Rrj​∈R for product jjj, the (Assortment) problem is max⁡S⊆N∑j∈NPj,Srj\max_{S\subseteq N}\sum_{j\in N}P_{j,S}r_jmaxS⊆N​∑j∈N​Pj,S​rj​. Two linear programs enter: the maximization of ∑jrjxj\sum_j r_jx_j∑j​rj​xj​ over the polyhedron

H={(x,z)∈R+2n: xj+zj=λj+∑i∈Nρi,jzi  ∀j∈N},\mathcal H=\Big\{(x,z)\in\mathbb R^{2n}_+:\ x_j+z_j=\lambda_j+\sum_{i\in N}\rho_{i,j}z_i\ \ \forall j\in N\Big\},H={(x,z)∈R+2n​: xj​+zj​=λj​+i∈N∑​ρi,j​zi​  ∀j∈N},

and its dual

min⁡v∈Rn{∑j∈Nλjvj: vj≥rj  ∀j∈N,  vj≥∑i∈Nρj,ivi  ∀j∈N}.(Dual)\min_{v\in\mathbb R^n}\Big\{\sum_{j\in N}\lambda_jv_j:\ v_j\ge r_j\ \ \forall j\in N,\ \ v_j\ge\sum_{i\in N}\rho_{j,i}v_i\ \ \forall j\in N\Big\}.\qquad\text{(Dual)}v∈Rnmin​{j∈N∑​λj​vj​: vj​≥rj​  ∀j∈N,  vj​≥i∈N∑​ρj,i​vi​  ∀j∈N}.(Dual)

Formalization targets

Goal: Theorem 2 (p. 1326)

For every optimal solution v^\hat vv^ of (Dual), the set S^={j∈N:v^j=rj}\hat S=\{j\in N:\hat v_j=r_j\}S^={j∈N:v^j​=rj​} is an optimal assortment:

∑j∈NPj,S rj ≤ ∑j∈NPj,S^ rjfor all S⊆N.\sum_{j\in N}P_{j,S}\,r_j\ \le\ \sum_{j\in N}P_{j,\hat S}\,r_j\qquad\text{for all } S\subseteq N .j∈N∑​Pj,S​rj​ ≤ j∈N∑​Pj,S^​rj​for all S⊆N.

Milestones, in the order the paper uses them

  1. (p. 1325) The (Balance) equations have a unique nonnegative solution for every SSS.
  2. Lemma 1 (p. 1326): for an extreme point (x^,z^)(\hat x,\hat z)(x^,z^) of H\mathcal HH and Sx^={j:x^j>0}S_{\hat x}=\{j:\hat x_j>0\}Sx^​={j:x^j​>0}, Pj,Sx^=x^jP_{j,S_{\hat x}}=\hat x_jPj,Sx^​​=x^j​ and Rj,Sx^=z^jR_{j,S_{\hat x}}=\hat z_jRj,Sx^​​=z^j​ for all jjj.
  3. (p. 1326) The linear program over H\mathcal HH has an optimal solution, and its optimal value equals the optimal value of (Assortment).
  4. (p. 1326) (Dual) has an optimal solution, and its optimal value equals that of the linear program over H\mathcal HH.
  5. (p. 1326, proof of Theorem 2) An optimal v^\hat vv^ satisfies v^j=rj\hat v_j=r_jv^j​=rj​ or v^j=∑iρj,iv^i\hat v_j=\sum_i\rho_{j,i}\hat v_iv^j​=∑i​ρj,i​v^i​ for each jjj.

Significance

Theorem 2 reduces a combinatorial problem over 2n2^n2n offer sets to a linear program, so the assortment problem under the Markov chain choice model is solvable in polynomial time. The same structure drives the rest of the paper: the dual variables v^j\hat v_jv^j​ are used to show that optimal offer sets shrink when all revenues fall by a common amount, to show that the optimal offer sets of the single-resource dynamic program are nested in the remaining capacity and time, and to reduce the choice-based network linear program to a compact one.

The result is proved in the paper. This mission produces a machine-checked version of it and of its supporting lemmas; to the knowledge of the mission author no formalization of the Markov chain choice model exists. The definition layer (the model, the (Balance) solution, H\mathcal HH and (Dual)) is the first formal encoding of this choice model and is shared, with the same encoding, by missions II–IV.

Difficulty

The obvious route, comparing ∑jPj,Srj\sum_jP_{j,S}r_j∑j​Pj,S​rj​ across offer sets directly, fails because Pj,SP_{j,S}Pj,S​ is defined only implicitly through a linear system whose coefficient matrix changes with SSS; there is no closed-form expression that can be compared across offer sets, and enumerating the 2n2^n2n sets is exponential. The connection to linear programming needs an exact correspondence between the vertices of H\mathcal HH and the (Balance) solutions, which is a statement about polyhedra, not about Markov chains. Even well-posedness is not free: existence, uniqueness and nonnegativity of (PS,RS)(P_S,R_S)(PS​,RS​) depend on the row sums ∑iρj,i\sum_i\rho_{j,i}∑i​ρj,i​ being strictly below one. Mathlib has extreme points of convex sets but no ready-made theory of vertices of polyhedra or of linear programming duality in this form.

Formalization scope

Products are Fin n (0-based), offer sets are Finset (Fin n) (the paper's S⊂NS\subset NS⊂N is non-strict inclusion, so every subset including ∅\emptyset∅ and NNN is an offer set), and rho j i is ρj,i\rho_{j,i}ρj,i​, the transition from jjj to iii. The model structure carries the paper's standing assumptions λj>0\lambda_j>0λj​>0 and ∑iρj,i<1\sum_i\rho_{j,i}<1∑i​ρj,i​<1 (p. 1325) and the implicit nonnegativity ρj,i≥0\rho_{j,i}\ge0ρj,i​≥0. Revenues are arbitrary reals with no sign assumption. R2n\mathbb R^{2n}R2n is (Fin n → ℝ) × (Fin n → ℝ), and extreme points are Mathlib's Set.extremePoints ℝ.

The pair (PS,RS)(P_S,R_S)(PS​,RS​) is a solution of (Balance) chosen by Classical.epsilon; milestone 1 states that it satisfies (Balance), is nonnegative and equals every solution, so all statements are about the paper's (PS,RS)(P_S,R_S)(PS​,RS​) and not about a junk value. Every optimum (of (Assortment), of the linear program over H\mathcal HH, of (Dual)) is stated as "feasible and at least as good as every feasible point", never through sSup or sInf. The goal is stated for every optimal v^\hat vv^ of (Dual), which is what the proof uses; uniqueness of v^\hat vv^ is neither assumed nor claimed. Replacing the hypothesis "v^\hat vv^ optimal for (Dual)" by "v^\hat vv^ feasible for (Dual)" would make the statement false, and an existential over v^\hat vv^ would weaken it; neither is the target.

No hypothesis beyond the paper's is added. Two printed index slips on pp. 1325–1326 (a garbled sum in the discussion after (Balance), and ∑iρi,jz^j\sum_i\rho_{i,j}\hat z_j∑i​ρi,j​z^j​ for ∑iρi,jz^i\sum_i\rho_{i,j}\hat z_i∑i​ρi,j​z^i​ in the proof of Lemma 1) are not part of any statement here.

Welcome contributions: the existence and uniqueness of (Balance) solutions via Neumann series for substochastic matrices, a characterization of vertices of polyhedra given by equality constraints and nonnegativity, and a strong duality statement usable for this primal–dual pair. These are reusable well beyond this mission.

Selected references

  • J. B. Feldman, H. Topaloglu, Revenue Management Under the Markov Chain Choice Model, Operations Research 65(5):1322–1342, 2017. https://doi.org/10.1287/opre.2017.1628
  • J. Blanchet, G. Gallego, V. Goyal, A Markov Chain Approximation to Choice Modeling, Operations Research 64(4):886–905, 2016. https://doi.org/10.1287/opre.2016.1505
  • M. L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming, Wiley, 1994. https://doi.org/10.1002/9780470316887
8 thms2 active usersReviewed
Convex OptimizationDiscrete GeometryOptimization·Captain: Shuze Chen

Discrete Convex Analysis IX: The Discrete Conjugacy TheoremTextbook

Motivation

The Legendre-Fenchel transform is the single most structurally important operation in convex analysis: for a proper closed convex function fff, its conjugate f∙(p)=sup⁡x{⟨p,x⟩−f(x)}f^\bullet(p) = \sup_x \{\langle p,x\rangle - f(x)\}f∙(p)=supx​{⟨p,x⟩−f(x)} is again proper closed convex, and the transform is an involution — f∙∙=ff^{\bullet\bullet} = ff∙∙=f. This one fact underlies duality theory across optimization: every strong-duality theorem is, at bottom, a statement about conjugate pairs. Chapters 6 and 7 of this book developed M-convex and L-convex functions as if they were two separate theories, each with its own exchange axiom, optimality criterion, and proximity theorem. Chapter 8 reveals they were never separate: the Legendre-Fenchel transform, suitably discretized, is a bijection between the two classes. This mission formalizes that discrete conjugacy theorem together with its classical real-valued precursor and a genuine function-level generalization of Edmonds's intersection theorem, completing the picture that chunks 06 through 09 built the two halves of.

Setting

Let VVV be a finite ground set. For f:RV→R∪{+∞}f : \mathbb R^V \to \mathbb R \cup \{+\infty\}f:RV→R∪{+∞}, the Legendre-Fenchel transform is f∙(p)=sup⁡{⟨p,x⟩−f(x):x∈RV}f^\bullet(p) = \sup\{\langle p,x\rangle - f(x) : x \in \mathbb R^V\}f∙(p)=sup{⟨p,x⟩−f(x):x∈RV}; fff is submodular if f(x)+f(y)≥f(x∨y)+f(x∧y)f(x)+f(y) \ge f(x\vee y)+f(x\wedge y)f(x)+f(y)≥f(x∨y)+f(x∧y) and supermodular under the reverse inequality. For f:ZV→R∪{+∞}f : \mathbb Z^V \to \mathbb R \cup \{+\infty\}f:ZV→R∪{+∞}, the discrete Legendre-Fenchel transform restricts the same supremum formula to p∈ZVp \in \mathbb Z^Vp∈ZV: f∙(p)=sup⁡{⟨p,x⟩−f(x):x∈ZV}f^\bullet(p) = \sup\{\langle p,x\rangle - f(x) : x \in \mathbb Z^V\}f∙(p)=sup{⟨p,x⟩−f(x):x∈ZV} for p∈ZVp \in \mathbb Z^Vp∈ZV — a genuinely different object from the real-valued transform, since the supremum is now over integer xxx only, and the codomain is checked back against the discrete M-/L-convexity axioms of chunks 06–09. The integer biconjugate f∙∙f^{\bullet\bullet}f∙∙ is the transform applied twice. fff is integer valued if every finite value it takes is an integer (the classes M[Z→Z]M[\mathbb Z\to\mathbb Z]M[Z→Z], L[Z→Z]L[\mathbb Z\to\mathbb Z]L[Z→Z] of the goal theorem are exactly the M-/L-convex functions with this property).

Formalization targets

Goal: Theorem 8.12 (the discrete conjugacy theorem)

(1) The classes M[Z→Z]M[\mathbb Z\to\mathbb Z]M[Z→Z] and L[Z→Z]L[\mathbb Z\to\mathbb Z]L[Z→Z] are in one-to-one correspondence under the discrete Legendre-Fenchel transform: for f∈M[Z→Z]f \in M[\mathbb Z\to\mathbb Z]f∈M[Z→Z] and g∈L[Z→Z]g \in L[\mathbb Z\to\mathbb Z]g∈L[Z→Z], f∙∈L[Z→Z]f^\bullet \in L[\mathbb Z\to\mathbb Z]f∙∈L[Z→Z], g∙∈M[Z→Z]g^\bullet \in M[\mathbb Z\to\mathbb Z]g∙∈M[Z→Z], f∙∙=ff^{\bullet\bullet}=ff∙∙=f, and g∙∙=gg^{\bullet\bullet}=gg∙∙=g. (2) The same correspondence holds between M♮[Z→Z]M^\natural[\mathbb Z\to\mathbb Z]M♮[Z→Z] and L♮[Z→Z]L^\natural[\mathbb Z\to\mathbb Z]L♮[Z→Z].

Milestones: Theorem 8.1, Proposition 8.11, Theorem 8.17

Theorem 8.1: the conjugate of a real-valued submodular function is always supermodular — the classical warm-up, and evidence that submodularity/supermodularity is not symmetric under conjugation on its own (the converse fails). Proposition 8.11: the integer biconjugate recovers fff at any point with a nonempty integer subdifferential — the fact that makes discrete biconjugation meaningful at all. Theorem 8.17 (the M-convex intersection theorem): a point jointly minimizes a sum of two M♮^\natural♮-convex functions if and only if a single linear functional separately certifies it as a minimizer of each perturbed function — the function-level generalization of chunk 04's Edmonds's intersection theorem for M-convex sets.

Significance

The result itself. The discrete conjugacy theorem is, in the book's own words, "the unifying result of the entire book": every theorem proved separately for M-convex functions (chunks 06–07) has an exact mirror for L-convex functions (chunks 08–09) precisely because the Legendre-Fenchel transform carries one class to the other. Theorem 8.17's function-level Edmonds generalization shows the payoff directly — the classical matroid-intersection-style min-max duality of chunk 04 was never really about sets; it is a special case (indicator functions) of a duality that holds for the whole class of M-convex functions.

Formalizing it. No matching item exists on the platform for conjugate functions, discrete conjugacy, or this generality of intersection theorem. This mission gives the first formal statement of the discrete conjugacy theorem, distinguishing it carefully from its real-valued (polyhedral) precursor, Theorem 8.4 — a genuinely different, harder theorem this mission does not draft (see Formalization scope), since the integer bijection needs the M-/L-proximity theorems of chunks 06–09 to control integrality under convex extension, while the real-valued case does not.

Difficulty

The obvious approach — try to prove the discrete conjugacy theorem directly by mimicking the real-valued proof (Theorem 8.4) with ℤ in place of ℝ everywhere — fails, because the real-valued proof's key step (Proposition 8.3, an infimal-convolution argument comparing arg min sets of perturbed polyhedral functions) has no immediate discrete analogue: a discrete arg min need not vary continuously with the perturbation the way a polyhedral one does. The book's actual strategy instead routes through the convex extension of the discrete function (chunk 06/08's bridge to chapter 3's integral convexity), applies the already-proved real-valued conjugacy theorem to the extension, and then must separately argue that the resulting conjugate, restricted back to integer points, is again integer-valued and satisfies the discrete exchange axiom — an argument that needs different treatment depending on whether the original function's domain is bounded or unbounded (an exhaustion argument via restriction to a growing integer interval, invoking chunk 06's proximity theorem to control convergence). Skipping this discreteness argument and treating the real-valued theorem as if it settled the integer case would silently discard exactly the chapter's own point.

Formalization scope

The ground set VVV is a Fintype with DecidableEq. ConvexConjugate (the discrete transform) has domain and codomain both (V → ℤ) → WithTop ℝ, obtained by taking the defining supremum in EReal (a complete lattice, so it is always total) and projecting back via a new FromEReal map — this is what lets the biconjugate f•• typecheck as an equality of functions of the same type as f. ConvexConjugateR (the real-valued transform, used only by the milestone Theorem 8.1) is a separate object with no shared code, per the explicit warning against conflating the two transforms; the two never appear in the same item.

A trivializing formalization of the goal would draft only the real-valued case (Theorem 8.4) as if it were the discrete theorem, or would silently allow WithTop ℝ's subtraction-avoidance convention to change which values are compared; neither is done. Theorem 8.4 itself (the polyhedral conjugacy theorem) is not drafted in this mission at all — it would require a fresh, otherwise-unused polyhedral M-/L-convex-function layer on Rⱽ that no other item here needs (see MODERATION_NOTES.md). The M-/L-separation theorems (8.15, 8.16) and the Fenchel-type duality theorem (8.21) are likewise left for a follow-on mission; contributions building the polyhedral bridge or the separation theorems, which depend on machinery this mission establishes, are welcome.

Selected references

  • K. Murota, Discrete Convex Analysis, SIAM, 2003. DOI: 10.1137/1.9780898718508.
13 thms2 active usersReviewed
🏆Completed
CombinatoricsGraph TheoryOptimization+1·Captain: mikedeng1

An Analysis of Several Heuristics for the Traveling Salesman Problem IV: Insertion Heuristics Can Return Poor k-Optimal ToursResearch Paper

Motivation

Insertion heuristics build a traveling salesman tour one city at a time: start from a single city, and at each step choose a city not yet on the subtour and splice it into the subtour where it lengthens the subtour least. Local search heuristics start from a tour and repeatedly replace a few of its edges by others while this shortens the tour. Both families are standard in practice, and a natural engineering idea is to combine them: run an insertion heuristic, then polish the result by local search. The question this mission formalizes is whether local optimality of the insertion tour certifies anything about its quality.

Rosenkrantz, Stearns and Lewis (SIAM J. Comput. 6(3), 1977) answered this for graphs satisfying the triangle inequality. Their §4 proves that nearest and cheapest insertion always return a tour of length at most 2(1−1/n)2(1-1/n)2(1−1/n) times the optimal length, and their Theorem 5 shows this bound is attained. Their §7 then shows that the very tour attaining the bound is kkk-optimal for every k≤n/4k\le n/4k≤n/4: no exchange of kkk edges shortens it. So the insertion bound is tight even for tours that local search with kkk-changes cannot improve.

Timeline, as far as this mission is concerned:

  • 1965: Lin (Bell System Tech. J. 44) defines kkk-optimal tours and uses 3-optimal local search.
  • 1973: Lin and Kernighan (Oper. Res. 21) generalize the edge-exchange neighbourhoods.
  • 1977: Rosenkrantz, Stearns and Lewis prove the 2(1−1/n)2(1-1/n)2(1−1/n) upper bound for nearest and cheapest insertion (Theorem 4 and its corollary), its tightness for n≥6n\ge 6n≥6 (Theorem 5), the existence of kkk-optimal tours with the same ratio (Theorem 6, stated for n≥8n\ge 8n≥8), and the Corollary combining the two.

Setting

A traveling salesman graph on nnn nodes is the node set N={1,…,n}N=\{1,\dots,n\}N={1,…,n} with a distance d(i,j)≥0d(i,j)\ge 0d(i,j)≥0 that is symmetric and satisfies the triangle inequality d(i,k)≤d(i,j)+d(j,k)d(i,k)\le d(i,j)+d(j,k)d(i,k)≤d(i,j)+d(j,k). A tour is a Hamiltonian circuit; its length is the sum of its edge lengths; OPTIMAL is the least length of a tour. As in the paper, the identically zero distance is excluded, so OPTIMAL >0>0>0.

A subtour is a circuit on a subset of the nodes (a single node is a subtour without edges). For a subtour TTT and a node k∉Tk\notin Tk∈/T, TOUR(T,k)(T,k)(T,k) is obtained by deleting an edge (x,y)(x,y)(x,y) of TTT minimizing d(x,k)+d(k,y)−d(x,y)d(x,k)+d(k,y)-d(x,y)d(x,k)+d(k,y)−d(x,y) and adding (x,k)(x,k)(x,k) and (k,y)(k,y)(k,y); COST(T,k)(T,k)(T,k) is the resulting increase in length. An insertion method chooses nodes a0,a1,…,an−1a_0,a_1,\dots,a_{n-1}a0​,a1​,…,an−1​, starts from T1={a0}T_1=\{a_0\}T1​={a0​} and sets Ti+1=TOUR(Ti,ai)T_{i+1}=\mathrm{TOUR}(T_i,a_i)Ti+1​=TOUR(Ti​,ai​); INSERT is the length of TnT_nTn​. Nearest insertion chooses aia_iai​ minimizing d(Ti,x)=min⁡y∈Tid(y,x)d(T_i,x)=\min_{y\in T_i}d(y,x)d(Ti​,x)=miny∈Ti​​d(y,x) over x∉Tix\notin T_ix∈/Ti​; cheapest insertion chooses aia_iai​ minimizing COST(Ti,x)(T_i,x)(Ti​,x). Ties are broken arbitrarily.

A kkk-change of a tour deletes kkk of its edges and adds kkk other edges so that another tour is obtained. A tour is kkk-optimal if no kkk-change produces a strictly shorter tour.

The extremal instance is the circle (Nn,dn)(N_n,d_n)(Nn​,dn​): nnn cities equally spaced on a circular road, with dn(i,j)d_n(i,j)dn​(i,j) the smallest m≥0m\ge 0m≥0 with i−j≡mi-j\equiv mi−j≡m or j−i≡m(modn)j-i\equiv m \pmod nj−i≡m(modn). The insertion run of Theorem 5 inserts the cities in the order 1,2,…,n1,2,\dots,n1,2,…,n and produces the zig-zag tour TnT_nTn​: city 1, then the even cities in increasing order, then the odd cities in decreasing order.

Formalization targets

Goal: the Corollary to Theorem 6

For n≥6n\ge 6n≥6 and 4k≤n4k\le n4k≤n there is a traveling salesman graph with OPTIMAL >0>0>0 on which some run of nearest insertion, and some run of cheapest insertion, return a kkk-optimal tour with

INSERTOPTIMAL=2(1−1n).\frac{\mathrm{INSERT}}{\mathrm{OPTIMAL}}=2\left(1-\frac1n\right).OPTIMALINSERT​=2(1−n1​).

Milestones

  1. The insertion run on the circle: the subtours TiT_iTi​ and nodes ai=i+1a_i=i+1ai​=i+1 form an insertion run that obeys both the nearest and the cheapest rule (proof of Theorem 5).
  2. On the circle, TnT_nTn​ has length 2(n−1)2(n-1)2(n−1) and OPTIMAL =n=n=n (proof of Theorem 5).
  3. Theorem 5: for n≥6n\ge 6n≥6 there is a graph with INSERT/OPTIMAL =2(1−1/n)=2(1-1/n)=2(1−1/n) for both methods.
  4. Equation (7.4): the length of a tour of the circle is the sum over unit edges eee of COUNT(e,T)(e,T)(e,T), the number of times eee is traversed when each tour edge is replaced by a shortest arc.
  5. Every tour of the circle is odd or even (all counts of one parity), eq. (7.5).
  6. TnT_nTn​ is the shortest even tour, so every tour shorter than TnT_nTn​ is odd.
  7. TnT_nTn​ is kkk-optimal for every k≤n/4k\le n/4k≤n/4.
  8. Theorem 6: for n≥8n\ge 8n≥8 there is a graph with a tour that is kkk-optimal for all k≤n/4k\le n/4k≤n/4 and has LOCALOPT/OPTIMAL =2(1−1/n)=2(1-1/n)=2(1−1/n).

Significance

The result. The Corollary shows that the 2(1−1/n)2(1-1/n)2(1−1/n) worst-case guarantee of nearest and cheapest insertion cannot be improved by requiring that the returned tour survive kkk-change local search, for kkk up to a quarter of the number of cities. Theorem 6 says more generally that kkk-optimality with k≤n/4k\le n/4k≤n/4 does not bound the ratio to the optimum below 2(1−1/n)2(1-1/n)2(1−1/n). Together with the paper's upper bound, the insertion guarantee is exact, and it stays exact after local polishing with small neighbourhoods.

Formalizing it. All statements are proved in the paper; none, to our knowledge, has been machine-checked. The platform has an upper bound for nearest insertion (SupplyChainTheory, Theorem 10.7, ratio at most 2) but no tightness example and no notion of kkk-optimality. This mission produces a reusable definition of kkk-changes against arbitrary tours, the circle metric, and the parity-counting argument on a cycle, and it supplies the calculations the paper omits ("We omit these calculations but note that they require the assumption n≥6n\ge 6n≥6").

Difficulty

Two steps carry the weight. First, the omitted calculations for the insertion run: at each stage one must show that inserting aia_iai​ between i−1i-1i−1 and iii minimizes the insertion increase over every edge of the zig-zag subtour, and that no other outside city can be inserted for less than 2; the claim fails for n=4n=4n=4 and n=5n=5n=5, so the verification must use n≥6n\ge 6n≥6 in an essential way. Second, kkk-optimality is a statement about every tour at edge difference kkk, not about 2-opt segment reversals or any specific move family. A search over moves of a special form does not establish it; the argument must bound the length of an arbitrary tour at edge difference kkk from below.

Formalization scope

Nodes are Fin n, so the paper's node mmm is index m−1m-1m−1 and ai=i+1a_i=i+1ai​=i+1 is index iii; subtour indices stay 1-based (T1=[a0]T_1=[a_0]T1​=[a0​], approximation TnT_nTn​). A distance is d : Fin n → Fin n → ℝ with the structure IsTSPDist (symmetric, nonnegative, triangle inequality, and d(i,i)=0d(i,i)=0d(i,i)=0; the last is a normalization absent from the paper that changes no length). A tour is an Equiv.Perm (Fin n), a subtour a list read cyclically; OPTIMAL is Finset.univ.inf' over permutations, the true minimum. TOUR(T,k)(T,k)(T,k) is insertion at a position minimizing the new length; COST is a minimum over positions; the nearest-insertion distance (4.1) takes values in WithTop ℝ, so no junk value arises. kkk-optimality compares the tour with every permutation whose edge set (unordered pairs) misses exactly kkk of the tour's edges. Ratios are multiplied out.

COUNT(e,T)(e,T)(e,T) needs a choice of shortest arc for antipodal pairs when nnn is even; the formalization takes the arc through min⁡(x,y),…,max⁡(x,y)\min(x,y),\dots,\max(x,y)min(x,y),…,max(x,y). The paper's argument does not depend on this choice.

Deviations from the printed text: the Corollary is stated for n≥6n\ge 6n≥6 (the printed statement says only 4k≤n4k\le n4k≤n, but its proof uses the example of Theorem 5, which exists for n≥6n\ge 6n≥6; for k=0k=0k=0, n=3n=3n=3 the printed statement is false). Theorem 6 keeps its printed n≥8n\ge 8n≥8. In the proof of Theorem 5 the paper writes "(4.2) holds" where the cheapest-insertion condition (4.3) is meant; the formal statement uses (4.3).

The existence statements carry OPTIMAL >0>0>0, the paper's standing assumption (1.1). Without it, the zero distance would make every length zero and every tour kkk-optimal, which would satisfy the ratio equations trivially; that formalization is ruled out.

Contributions welcome: proofs of any milestone, in particular the omitted insertion calculations and the parity lemma, and reusable lemmas on cyclic lists and edge sets of permutations.

Selected references

  • D. J. Rosenkrantz, R. E. Stearns, P. M. Lewis II, An Analysis of Several Heuristics for the Traveling Salesman Problem, SIAM J. Comput. 6(3):563–581, 1977. https://doi.org/10.1137/0206041
  • S. Lin, Computer solutions of the traveling salesman problem, Bell System Tech. J. 44:2245–2269, 1965. https://doi.org/10.1002/j.1538-7305.1965.tb04146.x
  • S. Lin, B. W. Kernighan, An effective heuristic algorithm for the traveling-salesman problem, Oper. Res. 21(2):498–516, 1973. https://doi.org/10.1287/opre.21.2.498
14 thms2 active usersReviewed
🏆Completed
Algorithmic Game TheoryOptimization·Captain: mikedeng1

Approximation Algorithms for Combinatorial Auctions with Complement-Free Bidders III: A Greedy Price-Update Algorithm Is a 2-Approximation for XOS BiddersResearch Paper

Motivation

In a combinatorial auction a set of mmm items is sold to nnn bidders who value bundles of items rather than single items. Spectrum auctions, procurement of transportation lanes and the allocation of cloud resources all have this form, and the central algorithmic question is how to allocate the items so as to maximize the social welfare, the sum of the bidders' values for what they receive. Even to describe a general valuation takes 2m2^m2m numbers, so algorithms access the bidders through queries, and the achievable approximation depends on the class of valuations allowed.

Dobzinski, Nisan and Schapira (Math. Oper. Res. 35(1), 2010) study bidders without complementarities. For the class of XOS valuations, maxima of additive valuations, which strictly contains the submodular valuations, they give LP-based algorithms and, in §3.3, a purely combinatorial algorithm: bidders arrive one at a time, take their demanded bundle at the current item prices, and raise the prices of what they took. Its analysis charges the welfare of any allocation to the item prices the algorithm sets, an argument that uses only the demand and XOS oracles.

Setting

The items are M={1,…,m}M=\{1,\dots,m\}M={1,…,m} and the bidders 1,…,n1,\dots,n1,…,n.

  • An additive valuation (a clause) www assigns nonnegative values w(1),…,w(m)w(1),\dots,w(m)w(1),…,w(m) to the items and w(S)=∑j∈Sw(j)w(S)=\sum_{j\in S}w(j)w(S)=∑j∈S​w(j) to a bundle S⊆MS\subseteq MS⊆M.
  • An XOS valuation vvv is given by a nonempty finite set of clauses {w1,…,wt}\{w_1,\dots,w_t\}{w1​,…,wt​} (its XOS expression) through v(S)=max⁡kwk(S)v(S)=\max_k w_k(S)v(S)=maxk​wk​(S). A clause wkw_kwk​ with wk(S)=v(S)w_k(S)=v(S)wk​(S)=v(S) is a maximizing clause for SSS. Every XOS valuation is normalized, v(∅)=0v(\emptyset)=0v(∅)=0, and monotone.
  • An allocation is a tuple of pairwise disjoint bundles O1,…,OnO_1,\dots,O_nO1​,…,On​; items may stay unallocated. Its welfare is ∑ivi(Oi)\sum_i v_i(O_i)∑i​vi​(Oi​).
  • A demand oracle for bidder iii answers, given item prices p∈Rmp\in\mathbb R^mp∈Rm, a bundle maximizing vi(T)−∑j∈Tpjv_i(T)-\sum_{j\in T}p_jvi​(T)−∑j∈T​pj​. An XOS oracle answers, given a bundle SSS, a maximizing clause for SSS in viv_ivi​.

The greedy price-update algorithm. Start with all bundles empty and all prices pj=0p_j=0pj​=0. For i=1,…,ni=1,\dots,ni=1,…,n: let SiS_iSi​ be bidder iii's demand at the current prices; remove the items of SiS_iSi​ from the bundles of the earlier bidders; let qiq^iqi be the maximizing clause for SiS_iSi​ in viv_ivi​; set pj=qjip_j=q^i_jpj​=qji​ for j∈Sij\in S_ij∈Si​. Write pkp^kpk for the prices after stage kkk (p0=0p^0=0p0=0), pk(T)=∑j∈Tpjkp^k(T)=\sum_{j\in T}p^k_jpk(T)=∑j∈T​pjk​, and A1,…,AnA_1,\dots,A_nA1​,…,An​ for the final bundles.

In the Lean development these objects are XOSExpr, XOSExpr.val, IsAllocation, welfare, IsDemandOracle, IsXOSOracle, greedyState, greedyPrices and greedyAlloc in the namespace ComplementFreeCA.XOSGreedy.

Formalization targets

Goal: Theorem 3.3

For XOS valuations v1,…,vnv_1,\dots,v_nv1​,…,vn​, every demand oracle and every XOS oracle, and every allocation O1,…,OnO_1,\dots,O_nO1​,…,On​,

∑i=1nvi(Oi)  ≤  2∑i=1nvi(Ai).\sum_{i=1}^n v_i(O_i)\;\le\;2\sum_{i=1}^n v_i(A_i).i=1∑n​vi​(Oi​)≤2i=1∑n​vi​(Ai​).

The comparison with every allocation is the paper's comparison with the optimal allocation.

Milestones

  1. Lemma 3.4. The final prices are paid for by the algorithm's welfare:
pn(M)≤∑ivi(Ai).p^n(M)\le\sum_i v_i(A_i).pn(M)≤i∑​vi​(Ai​).
  1. Lemma 3.5. Prices never decrease: pjk≤pjk′p^k_j\le p^{k'}_jpjk​≤pjk′​ for all items jjj and stages k≤k′k\le k'k≤k′.
  2. Lemma 3.6. For every allocation OOO,
∑ivi(Oi)≤2 pn(M).\sum_i v_i(O_i)\le 2\,p^n(M).i∑​vi​(Oi​)≤2pn(M).

Three further statements are included without being milestones: the output A1,…,AnA_1,\dots,A_nA1​,…,An​ is an allocation (used implicitly by the paper); the first sentence of the proof of Lemma 3.6, that with Δi=pi(M)−pi−1(M)\Delta^i=p^i(M)-p^{i-1}(M)Δi=pi(M)−pi−1(M) one has Δi=max⁡T⊆M(vi(T)−pi−1(T))\Delta^i=\max_{T\subseteq M}\bigl(v_i(T)-p^{i-1}(T)\bigr)Δi=maxT⊆M​(vi​(T)−pi−1(T)); and the factor 222 is attained on the paper's two-item, two-bidder example (p. 9).

Significance

Theorem 3.3 shows that a factor-2 approximation of the optimal welfare for XOS bidders needs no linear program: one pass over the bidders, one demand query and one XOS query each. The paper's LP-based algorithm of §3.2 attains the better ratio e/(e−1)e/(e-1)e/(e−1), and Theorem 4.1 of the paper shows that for the larger class of complement-free bidders no (2−ϵ)(2-\epsilon)(2−ϵ)-approximation is possible with polynomial communication.

The result is proved in the paper; to the best of our knowledge it has no machine-checked proof. This mission produces a formal model of XOS valuations, demand and XOS oracles and the greedy run that is reusable for other price-based arguments, and a formal proof of the guarantee for every tie-breaking in both oracles.

Difficulty

The argument is elementary, but its bookkeeping is where a formal proof can go wrong. Items move between bidders: an item taken from an earlier bidder in step (b) is re-priced in step (d), and every priced item lies in exactly one final bundle. Lemma 3.4 needs this invariant across all nnn stages, together with the fact that a maximizing clause for the demanded set bounds viv_ivi​ on every subset, which holds because it is a clause of viv_ivi​ itself, not merely an additive function agreeing with viv_ivi​ on SiS_iSi​. Lemma 3.5 is a contradiction argument using optimality of the demand; Lemma 3.6 telescopes the price increases and needs prices to stay nonnegative. The argument must hold for arbitrary oracle answers, so no canonical demand can be assumed.

Formalization scope

  • Bidders are Fin n, items Fin m, bundles Finset (Fin m), values in ℝ. An XOS valuation is represented by its expression: a nonempty Finset (Fin m → ℝ) of clauses with nonnegative entries, evaluated with Finset.sup'. The paper's standing assumptions, normalized and monotone valuations (p. 1), hold automatically in this representation.
  • The oracles are function parameters dem : Fin n → (Fin m → ℝ) → Finset (Fin m) and cl : Fin n → Finset (Fin m) → (Fin m → ℝ) with hypotheses IsDemandOracle and IsXOSOracle; every theorem holds for every such pair, so no tie-breaking rule is fixed.
  • The run is a recursion on the stage: stage k+1k+1k+1 processes the 0-based bidder kkk, i.e. the paper's bidder k+1k+1k+1; after stage nnn the state is constant.
  • The paper's goal compares with the optimal allocation; the Lean goal compares with every allocation, which is equivalent and avoids an argmax. Stating the bound against one particular allocation, or against the algorithm's own output, would be trivial and is excluded.
  • There are no O(⋅)O(\cdot)O(⋅) constants: the factor 222 is the paper's.
  • Printed slip: the model on p. 1 writes an allocation as S1,…,SmS_1,\dots,S_mS1​,…,Sm​; it is one bundle per bidder, S1,…,SnS_1,\dots,S_nS1​,…,Sn​.
  • Out of scope: running time, the cost of simulating oracles (Proposition 2.1), and communication lower bounds.

Contributions welcome: proofs of the three milestones and of the goal, and reusable lemmas on the invariant that every priced item lies in exactly one final bundle.

Selected references

  • S. Dobzinski, N. Nisan, M. Schapira, Approximation Algorithms for Combinatorial Auctions with Complement-Free Bidders, Mathematics of Operations Research 35(1):1–13, 2010. https://doi.org/10.1287/moor.1090.0436
  • B. Lehmann, D. Lehmann, N. Nisan, Combinatorial auctions with decreasing marginal utilities, Games and Economic Behavior 55(2):270–296, 2006. https://doi.org/10.1016/j.geb.2005.02.006
6 thms2 active usersReviewed
🏆Completed
Algorithmic Game TheoryOptimizationProbability·Captain: mikedeng1

Approximation Algorithms for Combinatorial Auctions with Complement-Free Bidders II: Clause-Based Randomized Rounding for XOS BiddersResearch Paper

Motivation

In a combinatorial auction a seller offers mmm indivisible items to nnn bidders, each of whom values bundles of items rather than single items. Allocating the items so as to maximize the total value (the social welfare) is the central optimization problem of the area: it models spectrum auctions, procurement and resource allocation, and it is NP-hard and hard to approximate for general valuations. A large literature therefore studies restricted classes of valuations without complementarities. Among them the class XOS (valuations that are a maximum of additive valuations, also called fractionally subadditive) sits strictly between submodular and subadditive valuations and has become a standard benchmark class in algorithmic game theory.

Dobzinski, Nisan and Schapira (Math. Oper. Res. 35(1), 2010) gave, among other results, a randomized algorithm that approximates the optimal welfare for XOS bidders within a factor 1/(1−(1−1/n)n)1/(1-(1-1/n)^n)1/(1−(1−1/n)n), which is at most e/(e−1)≈1.582e/(e-1)\approx 1.582e/(e−1)≈1.582. The algorithm rounds the standard LP relaxation and resolves conflicts between bidders using the XOS structure. This mission formalizes that guarantee (Theorem 3.2 of the paper).

A short timeline: Lehmann, Lehmann and Nisan (EC 2001) introduced the XOS terminology and a 2-approximation for submodular bidders; the conference version of the present paper (STOC 2005) gave the e/(e−1)e/(e-1)e/(e−1) bound for XOS with demand and XOS oracles; Feige (STOC 2006) extended the e/(e−1)e/(e-1)e/(e−1) ratio to XOS bidders with demand oracles only and gave a 2-approximation for subadditive bidders.

Setting

Items are M={1,…,m}M=\{1,\dots,m\}M={1,…,m} and bidders are N={1,…,n}N=\{1,\dots,n\}N={1,…,n} with n≥1n\ge1n≥1. Bidder iii has a valuation viv_ivi​ assigning a real number vi(S)v_i(S)vi​(S) to every bundle S⊆MS\subseteq MS⊆M. An allocation is a tuple (O1,…,On)(O_1,\dots,O_n)(O1​,…,On​) of pairwise disjoint bundles; its welfare is ∑ivi(Oi)\sum_i v_i(O_i)∑i​vi​(Oi​).

A clause is an additive valuation www given by nonnegative item values w1,…,wmw_1,\dots,w_mw1​,…,wm​, with w(S)=∑j∈Swjw(S)=\sum_{j\in S}w_jw(S)=∑j∈S​wj​. A valuation vvv is XOS if there is a nonempty finite set WWW of clauses with

v(S)=max⁡w∈W ∑j∈Swj(S⊆M).v(S)=\max_{w\in W}\ \sum_{j\in S}w_j\qquad(S\subseteq M).v(S)=w∈Wmax​ j∈S∑​wj​(S⊆M).

A clause of WWW attaining the maximum for SSS is a maximizing clause for SSS in vvv; an XOS oracle returns one (arbitrarily, if several attain it).

The LP relaxation has a variable xi,Sx_{i,S}xi,S​ for every bidder iii and bundle SSS and asks to maximize OPT∗=∑i,Sxi,Svi(S)\mathrm{OPT}^*=\sum_{i,S}x_{i,S}v_i(S)OPT∗=∑i,S​xi,S​vi​(S) subject to ∑i∑S∋jxi,S≤1\sum_{i}\sum_{S\ni j}x_{i,S}\le1∑i​∑S∋j​xi,S​≤1 for each item jjj, ∑Sxi,S≤1\sum_S x_{i,S}\le1∑S​xi,S​≤1 for each bidder iii, and xi,S≥0x_{i,S}\ge0xi,S​≥0.

Randomized rounding draws a preallocation S1,…,SnS_1,\dots,S_nS1​,…,Sn​: independently for each bidder iii, bundle SSS is chosen with probability xi,Sx_{i,S}xi,S​ and the empty bundle with the remaining probability 1−∑Sxi,S1-\sum_S x_{i,S}1−∑S​xi,S​. The preallocation can give an item to several bidders.

The algorithm of §3.2: (i) draw a preallocation from an optimal LP solution xxx; (ii) let pi=(p1i,…,pmi)p^i=(p^i_1,\dots,p^i_m)pi=(p1i​,…,pmi​) be the maximizing clause for SiS_iSi​ in viv_ivi​; (iii) give each item jjj to a bidder iii with pji≥pji′p^i_j\ge p^{i'}_jpji​≥pji′​ for all i′i'i′. Write ALG\mathrm{ALG}ALG for the welfare of the resulting allocation.

Formalization targets

Goal: Theorem 3.2

For every XOS profile, every optimal LP solution xxx, every choice of maximizing clauses, every tie-breaking in step (iii) and every allocation OOO,

(1−(1−1n)n)∑ivi(Oi) ≤ E[ALG].\Big(1-\Big(1-\frac1n\Big)^n\Big)\sum_i v_i(O_i)\ \le\ \mathbb E[\mathrm{ALG}].(1−(1−n1​)n)i∑​vi​(Oi​) ≤ E[ALG].

Milestones

  1. ∑ivi(Oi)≤OPT∗\sum_i v_i(O_i)\le\mathrm{OPT}^*∑i​vi​(Oi​)≤OPT∗ for an optimal LP solution (step (i) of the proof of Theorem 3.1).
  2. Pointwise, ALG≥∑jQj\mathrm{ALG}\ge\sum_j Q_jALG≥∑j​Qj​ with Qj=max⁡ipjiQ_j=\max_i p^i_jQj​=maxi​pji​.
  3. Eq. (1): for 1≤k≤n1\le k\le n1≤k≤n and X1,…,Xk∈[0,1]X_1,\dots,X_k\in[0,1]X1​,…,Xk​∈[0,1] with ∑Xi≤1\sum X_i\le1∑Xi​≤1,
1−∏i≤k(1−Xi) ≥ 1−(1−∑Xik)k ≥ (1−(1−1k)k)∑Xi ≥ (1−(1−1n)n)∑Xi.1-\prod_{i\le k}(1-X_i)\ \ge\ 1-\Big(1-\tfrac{\sum X_i}{k}\Big)^k\ \ge\ \Big(1-\big(1-\tfrac1k\big)^k\Big)\sum X_i\ \ge\ \Big(1-\big(1-\tfrac1n\big)^n\Big)\sum X_i .1−i≤k∏​(1−Xi​) ≥ 1−(1−k∑Xi​​)k ≥ (1−(1−k1​)k)∑Xi​ ≥ (1−(1−n1​)n)∑Xi​.
  1. Lemma 3.3: E[Qj]≥(1−(1−1/n)n)∑i∑S∋jxi,S pj(i,S)\mathbb E[Q_j]\ge(1-(1-1/n)^n)\sum_i\sum_{S\ni j}x_{i,S}\,p^{(i,S)}_jE[Qj​]≥(1−(1−1/n)n)∑i​∑S∋j​xi,S​pj(i,S)​ for every feasible xxx.
  2. E[ALG]≥(1−(1−1/n)n) OPT∗(x)\mathbb E[\mathrm{ALG}]\ge(1-(1-1/n)^n)\,\mathrm{OPT}^*(x)E[ALG]≥(1−(1−1/n)n)OPT∗(x) for every feasible xxx.

Milestone 5 is stronger than the goal (it compares with the fractional value); the goal is stated against the integral optimum because that is what Theorem 3.2 asserts.

Significance

The bound 1−(1−1/n)n≥1−1/e1-(1-1/n)^n\ge1-1/e1−(1−1/n)n≥1−1/e is a constant-factor guarantee for a class that includes every submodular valuation, obtained from nothing more than the LP relaxation and the clause structure of XOS. It shows that the integrality gap of the configuration LP for XOS bidders is at most 1/(1−(1−1/n)n)1/(1-(1-1/n)^n)1/(1−(1−1/n)n), a fact reused in later work on welfare maximization, online allocation and posted-price mechanisms. The per-item analysis (Lemma 3.3 and Eq. (1)) is the same "1−1/e1-1/e1−1/e" correlation-gap argument that recurs in submodular maximization and prophet-inequality proofs.

The result is proved in the paper. To our knowledge it has no machine-checked proof. This mission produces a Lean statement and proof of the guarantee with the randomness made explicit, a reusable model of the configuration LP and of randomized rounding over bundles, and a formal version of the inequality in Eq. (1), which is independently useful.

Difficulty

Each piece of the argument is short; the work is in the bookkeeping. The preallocation is infeasible, so welfare cannot be read off from the rounding directly; the proof reduces it to per-item quantities QjQ_jQj​ and then lower-bounds E[Qj]\mathbb E[Q_j]E[Qj​] by comparing with a different assignment of item jjj that is not the algorithm's. The expectation of that auxiliary assignment involves a product of probabilities over bidders ordered by a conditional expectation, and the final bound needs the calculus inequality of Eq. (1) together with a summation by parts. A naive attempt to bound E[ALG]\mathbb E[\mathrm{ALG}]E[ALG] bidder by bidder fails, because a bidder's received bundle is not contained in its preallocated bundle, and the clause values of other bidders decide what it receives.

Formalization scope

  • Bidders are Fin n, items Fin m, bundles Finset (Fin m), valuations Finset (Fin m) → ℝ. XOS is stated through its expression: each viv_ivi​ comes with a nonempty finite clause set WiW_iWi​ of nonnegative clauses, and vi(S)v_i(S)vi​(S) is attained by a clause of WiW_iWi​ and bounded by all of them. Normalization and monotonicity, the paper's standing assumptions (p. 1), follow from this.
  • The LP has a variable for every bundle, including ∅\emptyset∅. "Optimal" is stated as feasible and not beaten by any feasible solution; existence of an optimum is not asserted.
  • The rounding is a finite product distribution over profiles σ:Fin n→\sigma:\mathrm{Fin}\,n\toσ:Finn→ bundles, with bidder iii's law qi(S)=xi,S+1[S=∅](1−∑Txi,T)q_i(S)=x_{i,S}+\mathbf 1[S=\emptyset](1-\sum_T x_{i,T})qi​(S)=xi,S​+1[S=∅](1−∑T​xi,T​); expectations are finite sums.
  • The XOS oracle is a function parameter with its specification, and step (iii) is any rule selecting a bidder with maximal clause value; every statement quantifies over all of them.
  • "Approximation" in Theorem 3.2 is read in expectation, as its proof establishes. There are no O(⋅)O(\cdot)O(⋅) constants in this mission.
  • Printed slip: the display of Lemma 3.3 (and the identity for OPT∗\mathrm{OPT}^*OPT∗ before it) sums xi,Spj(i,S)x_{i,S}p^{(i,S)}_jxi,S​pj(i,S)​ over all (i,S)(i,S)(i,S); the proof's final line restricts to S∋jS\ni jS∋j, and the printed version is false. The Lean states Lemma 3.3 with S∋jS\ni jS∋j.
  • Running time, the ellipsoid method and oracle complexity are out of scope.
  • Ruled out as trivializing: dropping the LP constraints on xxx (the item constraint is what makes Eq. (1) apply), stating the bound for a fixed preallocation instead of the expectation, or allowing negative clause values.

A complete development needs finite product distributions over bundles, the AM–GM inequality, monotonicity of (1−1/k)k(1-1/k)^k(1−1/k)k, and a summation-by-parts argument. The LP and rounding model are reusable for the other randomized-rounding results of the paper; proofs of any milestone are welcome.

Selected references

  • S. Dobzinski, N. Nisan, M. Schapira, Approximation Algorithms for Combinatorial Auctions with Complement-Free Bidders, Mathematics of Operations Research 35(1):1–13, 2010. https://doi.org/10.1287/moor.1090.0436
  • S. Dobzinski, N. Nisan, M. Schapira, Approximation algorithms for combinatorial auctions with complement-free bidders, STOC 2005. https://doi.org/10.1145/1060590.1060681
  • B. Lehmann, D. Lehmann, N. Nisan, Combinatorial auctions with decreasing marginal utilities, EC 2001. https://doi.org/10.1145/501158.501161
  • U. Feige, On maximizing welfare when utility functions are subadditive, STOC 2006. https://doi.org/10.1145/1132516.1132540
9 thms2 active usersReviewed
🏆Completed
Algorithmic Game TheoryOptimizationProbability·Captain: mikedeng1

Approximation Algorithms for Combinatorial Auctions with Complement-Free Bidders I: LP Rounding for Subadditive BiddersResearch Paper

Motivation

In a combinatorial auction a seller offers mmm indivisible items to nnn bidders, each of whom values bundles of items rather than single items. Allocating the items to maximize total value is the basic welfare problem of spectrum auctions, procurement and resource allocation, and it is the running example of algorithmic mechanism design. For general valuations no polynomial-time algorithm achieves a ratio polynomially better than m\sqrt mm​ under standard assumptions, so positive results require restricting the valuations. The most natural restriction is complement freeness (subadditivity): a bundle is never worth more than the sum of its parts.

Dobzinski, Nisan and Schapira (Math. Oper. Res. 35(1), 2010; conference version STOC 2005) gave the first polynomial-time algorithms with sub-polynomial approximation ratios for complement-free bidders given demand oracles. This mission formalizes their Section 3.1 algorithm, which rounds the linear-programming relaxation of the auction and splits the resulting infeasible solution into feasible ones.

Timeline. Lehmann, Lehmann and Nisan (2001) introduced the complement-free hierarchy and treated submodular bidders. The original version of the algorithm formalized here claimed an O(log⁡m)O(\log m)O(logm) ratio; Feige observed that the same algorithm achieves O(log⁡m/log⁡log⁡m)O(\log m/\log\log m)O(logm/loglogm) and that its ratio is at least Ω(log⁡m/log⁡log⁡m)\Omega(\sqrt{\log m/\log\log m})Ω(logm/loglogm​) (Feige, SIAM J. Comput. 39(1), 2009). Feige then obtained a constant ratio (222) for subadditive bidders by a different rounding.

Setting

Items are M={1,…,m}M=\{1,\dots,m\}M={1,…,m} and bidders N={1,…,n}N=\{1,\dots,n\}N={1,…,n}. Bidder iii has a valuation viv_ivi​ assigning a real value vi(S)v_i(S)vi​(S) to each bundle S⊆MS\subseteq MS⊆M. Every valuation is normalized, vi(∅)=0v_i(\emptyset)=0vi​(∅)=0, and monotone, S⊆T⇒vi(S)≤vi(T)S\subseteq T\Rightarrow v_i(S)\le v_i(T)S⊆T⇒vi​(S)≤vi​(T). A valuation is complement free if v(S∪T)≤v(S)+v(T)v(S\cup T)\le v(S)+v(T)v(S∪T)≤v(S)+v(T) for all S,TS,TS,T. An allocation is a tuple (S1,…,Sn)(S_1,\dots,S_n)(S1​,…,Sn​) of pairwise disjoint bundles; its welfare is ∑ivi(Si)\sum_i v_i(S_i)∑i​vi​(Si​), and OPTOPTOPT denotes the largest welfare.

The LP relaxation has a variable xi,S≥0x_{i,S}\ge0xi,S​≥0 for each bidder and bundle, with ∑i,S∋jxi,S≤1\sum_{i,S\ni j}x_{i,S}\le1∑i,S∋j​xi,S​≤1 for each item jjj and ∑Sxi,S≤1\sum_S x_{i,S}\le1∑S​xi,S​≤1 for each bidder iii; its objective is ∑i,Sxi,Svi(S)\sum_{i,S}x_{i,S}v_i(S)∑i,S​xi,S​vi​(S), with optimum OPT∗≥OPTOPT^*\ge OPTOPT∗≥OPT. Randomized rounding lets each bidder independently draw bundle SSS with probability xi,Sx_{i,S}xi,S​ and ∅\emptyset∅ with the remaining probability. The result, a preallocation, has expected welfare OPT∗OPT^*OPT∗ but may give an item to several bidders.

The algorithm takes k=⌊3log⁡m/log⁡log⁡m⌋k=\lfloor 3\log m/\log\log m\rfloork=⌊3logm/loglogm⌋ and:

  1. rounds until the preallocation (S1,…,Sn)(S_1,\dots,S_n)(S1​,…,Sn​) has every item in at most kkk bundles and ∑ivi(Si)≥OPT∗/3\sum_i v_i(S_i)\ge OPT^*/3∑i​vi​(Si​)≥OPT∗/3;
  2. splits each SiS_iSi​ into layers SirS_i^rSir​, r=1,…,kr=1,\dots,kr=1,…,k, where SirS_i^rSir​ holds the items of SiS_iSi​ that appear in exactly r−1r-1r−1 of S1,…,Si−1S_1,\dots,S_{i-1}S1​,…,Si−1​;
  3. picks the layer index rrr maximizing ∑ivi(Sir)\sum_i v_i(S_i^r)∑i​vi​(Sir​) and sets Ti=SirT_i=S_i^rTi​=Sir​;
  4. if some bidder has vi(M)≥∑i′vi′(Ti′)v_i(M)\ge\sum_{i'}v_{i'}(T_{i'})vi​(M)≥∑i′​vi′​(Ti′​), gives that bidder everything instead.

Formalization targets

Goal: Theorem 3.1, with the explicit ratio

For all sufficiently large mmm, for normalized, monotone, complement-free valuations and an optimal LP solution xxx with value OPT∗OPT^*OPT∗:

  1. if OPT∗>3max⁡ivi(M)OPT^*>3\max_i v_i(M)OPT∗>3maxi​vi​(M), one rounding meets the two conditions of step 1 with probability >1/6>1/6>1/6;
  2. from any preallocation meeting them, every admissible run of steps 2–4 outputs an allocation with
∑ivi(outputi) ≥ OPT3k,k=⌊3log⁡mlog⁡log⁡m⌋;\sum_i v_i(\text{output}_i)\ \ge\ \frac{OPT}{3k},\qquad k=\Big\lfloor\frac{3\log m}{\log\log m}\Big\rfloor;i∑​vi​(outputi​) ≥ 3kOPT​,k=⌊loglogm3logm​⌋;
  1. if OPT∗≤3max⁡ivi(M)OPT^*\le3\max_i v_i(M)OPT∗≤3maxi​vi​(M), the bidder maximizing vi(M)v_i(M)vi​(M) alone achieves OPT/3OPT/3OPT/3.

Milestones

  • OPT≤OPT∗OPT\le OPT^*OPT≤OPT∗ (proof, step (i)).
  • The layers of each index form an allocation and partition each SiS_iSi​ (proof, step (ii)).
  • Complement freeness gives ∑rvi(Sir)≥vi(Si)\sum_r v_i(S_i^r)\ge v_i(S_i)∑r​vi​(Sir​)≥vi​(Si​), and the best layer has welfare ≥OPT∗/(3k)\ge OPT^*/(3k)≥OPT∗/(3k) (proof, step (iii)).
  • Lemma 3.1: independent Bernoulli variables with ∑ipi≤1\sum_i p_i\le1∑i​pi​≤1 exceed 3log⁡m/log⁡log⁡m3\log m/\log\log m3logm/loglogm with probability ≤1/m2\le1/m^2≤1/m2.
  • Lemma 3.2: for XXX a sum of independent [0,1][0,1][0,1] variables with mean μ\muμ, Pr⁡[∣X−μ∣≥α]≤μ/α2\Pr[|X-\mu|\ge\alpha]\le\mu/\alpha^2Pr[∣X−μ∣≥α]≤μ/α2.
  • §3.1.1: some item appears more than 3log⁡m/log⁡log⁡m3\log m/\log\log m3logm/loglogm times with probability ≤1/m\le1/m≤1/m; the preallocation's welfare falls below OPT∗/3OPT^*/3OPT∗/3 with probability <3/4<3/4<3/4.

Significance

Theorem 3.1 was among the first polynomial-time approximation guarantees for welfare maximization with general subadditive bidders, and its layering argument is the standard way to turn an LP solution that is feasible up to a factor kkk into a feasible allocation losing only a factor kkk for subadditive objectives. The same argument applies to the kkk-duplicates auction and reappears in later rounding schemes. Lemma 3.1 is the standard balls-in-bins tail bound behind every log⁡m/log⁡log⁡m\log m/\log\log mlogm/loglogm load estimate.

The results are proved in the paper; none of them is formalized, on this platform or in Mathlib, as far as searches show. The mission produces a machine-checked version of the algorithm's guarantee with an explicit constant 3k3k3k in place of O(⋅)O(\cdot)O(⋅), a precise statement of the probabilistic step, and reusable statements of two concentration inequalities for sums of independent bounded variables.

Difficulty

The combinatorial part (steps (ii) and (iii)) is short. The difficulty is in step (i). The rounding is a product distribution over bundles, while the count of an item is a sum over bidders of indicators that depend on each bidder's whole bundle; connecting the finite product law to independent Bernoulli variables, and then to Lemma 3.1, requires building the independence structure explicitly. Lemma 3.1 itself does not follow from a Chernoff bound with a fixed relative deviation: the threshold 3log⁡m/log⁡log⁡m3\log m/\log\log m3logm/loglogm grows with mmm while the mean stays at most 111, and the bound must hold uniformly in the number of variables, which a fixed-deviation Chernoff statement does not give. Finally, the event-BBB bound needs the preallocation's welfare as a sum of independent variables in [0,1][0,1][0,1], which requires rescaling by max⁡ivi(M)\max_i v_i(M)maxi​vi​(M) and monotonicity.

Formalization scope

Bidders are Fin n, items Fin m, bundles Finset (Fin m), valuations Finset (Fin m) → ℝ. Normalization and monotonicity, the paper's standing assumptions (p. 1), are hypotheses of the goal. log⁡\loglog is the natural logarithm; the paper does not fix a base. The rounding law is written as explicit finite sums over profiles σ:Fin n→Finset (Fin m)\sigma:\texttt{Fin } n\to\texttt{Finset (Fin } m)σ:Fin n→Finset (Fin m) with product weights, so independence across bidders is literal; Lemmas 3.1 and 3.2 are stated measure-theoretically with Mathlib's iIndepFun.

Explicit constants and conventions that replace the paper's notation:

  • The ratio O(k)=O(log⁡m/log⁡log⁡m)O(k)=O(\log m/\log\log m)O(k)=O(logm/loglogm) is stated as 3k3k3k with k=⌊3log⁡m/log⁡log⁡m⌋k=\lfloor3\log m/\log\log m\rfloork=⌊3logm/loglogm⌋, the constant the proof establishes (step (iii), p. 6). "Sufficiently large mmm" is an existential m0m_0m0​.
  • The w.l.o.g. scaling max⁡ivi(M)=1\max_i v_i(M)=1maxi​vi​(M)=1 and the split at OPT∗=3OPT^*=3OPT∗=3 become the scale-free split at OPT∗=3max⁡ivi(M)OPT^*=3\max_i v_i(M)OPT∗=3maxi​vi​(M).
  • The choice of rrr in step (iii) and of the bidder in step (iv) are universally quantified over all admissible choices.

Printed slips resolved in the statements:

  • Lemma 3.1 mixes nnn and mmm. Here the number of variables is arbitrary, "sufficiently large" refers to mmm, and ∑ipi=1\sum_ip_i=1∑i​pi​=1 is relaxed to ∑ipi≤1\sum_ip_i\le1∑i​pi​≤1, which is what the application uses.
  • The display after Lemma 3.1 writes the threshold log⁡m/(3log⁡log⁡m)\log m/(3\log\log m)logm/(3loglogm); the lemma's 3log⁡m/log⁡log⁡m3\log m/\log\log m3logm/loglogm is used.
  • "Pr⁡[∨jEj]<1/n\Pr[\vee_jE_j]<1/nPr[∨j​Ej​]<1/n" and "≤1/n+3/4\le 1/n+3/4≤1/n+3/4" should read 1/m1/m1/m.
  • The event BBB is defined without its sum; it means ∑ivi(Si)<OPT∗/3\sum_iv_i(S_i)<OPT^*/3∑i​vi​(Si​)<OPT∗/3.

Trivializing formalizations are ruled out: the guarantee assumes the item-count bound (without it the layers do not cover the bundles), the probability statements assume an LP-feasible xxx, neither the layer index nor the step-(iv) bidder is fixed, and the ratio is the explicit 3k3k3k rather than an unspecified constant. Running time, the ellipsoid method and oracle complexity are out of scope. Contributions of general concentration lemmas for sums of independent bounded variables are welcome and reusable beyond this mission.

Selected references

  • S. Dobzinski, N. Nisan, M. Schapira, Approximation Algorithms for Combinatorial Auctions with Complement-Free Bidders, Mathematics of Operations Research 35(1):1–13, 2010. https://doi.org/10.1287/moor.1090.0436
  • U. Feige, On Maximizing Welfare When Utility Functions Are Subadditive, SIAM Journal on Computing 39(1):122–142, 2009. https://doi.org/10.1137/070680977
  • B. Lehmann, D. Lehmann, N. Nisan, Combinatorial Auctions with Decreasing Marginal Utilities, Games and Economic Behavior 55(2):270–296, 2006. https://doi.org/10.1016/j.geb.2005.02.006
  • M. Mitzenmacher, E. Upfal, Probability and Computing, Cambridge University Press, 2005. https://doi.org/10.1017/CBO9780511813603
12 thms2 active usersReviewed
🏆Completed
Convex OptimizationDiscrete GeometryOptimization·Captain: Shuze Chen

Discrete Convex Analysis VII: The L-Optimality Criterion and the Proximity TheoremTextbook

Motivation

Submodularity — the diminishing-returns property g(p)+g(q)≥g(p∨q)+g(p∧q)g(p) + g(q) \ge g(p \vee q) + g(p \wedge q)g(p)+g(q)≥g(p∨q)+g(p∧q) on a lattice — is one of the most useful structural hypotheses in combinatorial optimization, underlying efficient algorithms for network flows, matroid theory, and set-function minimization. Chapter 7 studies L-convex functions: functions on the integer lattice ZV\mathbb Z^VZV that are submodular and linear along the all-ones direction. This is the "dual" notion, under the conjugacy developed later in the book, to chunk 06's M-convex functions, and it inherits the same strong minimization theory — a purely local optimality criterion and a proximity theorem with an explicit distance bound — while additionally supporting a genuinely new characterization with no M-convex counterpart: discrete midpoint convexity, the direct lattice analogue of the classical real-valued midpoint convexity condition. This mission formalizes the chapter's definitional theorem, its midpoint-convexity characterization, the L-optimality criterion, and the L-proximity theorem itself.

Setting

Let VVV be a finite ground set. A function g:ZV→R∪{+∞}g : \mathbb Z^V \to \mathbb R \cup \{+\infty\}g:ZV→R∪{+∞} with nonempty effective domain is an L-convex function if it satisfies (SBF[Z]): g(p)+g(q)≥g(p∨q)+g(p∧q)g(p) + g(q) \ge g(p \vee q) + g(p \wedge q)g(p)+g(q)≥g(p∨q)+g(p∧q) for all p,qp, qp,q (∨,∧\vee, \wedge∨,∧ componentwise max/min), and (TRF[Z]): there is r∈Rr \in \mathbb Rr∈R with g(p+1)=g(p)+rg(p + \mathbf 1) = g(p) + rg(p+1)=g(p)+r for all ppp, where 1\mathbf 11 is the all-ones vector. An L♮^\natural♮-convex function is one whose lift to the extended ground set {0}∪V\{0\} \cup V{0}∪V is L-convex; equivalently (Theorem 7.1), ggg satisfies the translation-submodularity axiom (SBF♮^\natural♮[Z]): g(p)+g(q)≥g((p−α1)∨q)+g(p∧(q+α1))g(p) + g(q) \ge g((p - \alpha\mathbf 1) \vee q) + g(p \wedge (q + \alpha\mathbf 1))g(p)+g(q)≥g((p−α1)∨q)+g(p∧(q+α1)) for all p,qp, qp,q and all nonnegative integers α\alphaα. Discrete midpoint convexity asks g(p)+g(q)≥g(⌈(p+q)/2⌉)+g(⌊(p+q)/2⌋)g(p) + g(q) \ge g(\lceil (p+q)/2 \rceil) + g(\lfloor (p+q)/2 \rfloor)g(p)+g(q)≥g(⌈(p+q)/2⌉)+g(⌊(p+q)/2⌋) componentwise. For α\alphaα a positive integer, a point satisfies scaled local optimality if g(pα)≤g(pα±αχY)g(p_\alpha) \le g(p_\alpha \pm \alpha \chi_Y)g(pα​)≤g(pα​±αχY​) for every Y⊆VY \subseteq VY⊆V.

Formalization targets

Goal: Theorem 7.18 (the L-proximity theorem)

Assume α\alphaα is a positive integer and n=∣V∣n = |V|n=∣V∣. (1) If ggg is L-convex with g(p)=g(p+1)g(p) = g(p+\mathbf 1)g(p)=g(p+1) for all ppp, and pα∈dom⁡gp_\alpha \in \operatorname{dom} gpα​∈domg satisfies g(pα)≤g(pα+αχY)g(p_\alpha) \le g(p_\alpha + \alpha\chi_Y)g(pα​)≤g(pα​+αχY​) for all Y⊆VY \subseteq VY⊆V, then arg⁡min⁡g≠∅\arg\min g \ne \emptysetargming=∅ and there is p∗∈arg⁡min⁡gp^* \in \arg\min gp∗∈argming with the componentwise bound

pα≤p∗≤pα+(n−1)(α−1)1.p_\alpha \le p^* \le p_\alpha + (n-1)(\alpha-1)\mathbf 1.pα​≤p∗≤pα​+(n−1)(α−1)1.

(2) If ggg is L♮^\natural♮-convex and pαp_\alphapα​ satisfies the two-sided version, then there is p∗p^*p∗ with pα−n(α−1)1≤p∗≤pα+n(α−1)1p_\alpha - n(\alpha-1)\mathbf 1 \le p^* \le p_\alpha + n(\alpha-1)\mathbf 1pα​−n(α−1)1≤p∗≤pα​+n(α−1)1. The bound is a genuine vector (lattice-order) inequality, not an ℓ∞\ell^\inftyℓ∞-norm bound — the form later chapters' applications need.

Milestones: Theorems 7.1, 7.7, 7.14

Theorem 7.1: L♮^\natural♮-convexity (defined via the lift) is equivalent to the direct translation-submodularity axiom. Theorem 7.7: this same class is also characterized by discrete midpoint convexity — a three-way equivalence with the approach property (L♮^\natural♮-APR[Z]) as a bridge — giving L-convexity a genuinely different, more geometric face than anything available on the M-convex side. Theorem 7.14 (the L-optimality criterion): global optimality reduces to a purely local check against the sign-pattern neighbors p±χYp \pm \chi_Yp±χY​, mirroring chunk 06's Theorem 6.26 but with the plain L-convex case additionally requiring the periodicity condition g(p)=g(p+1)g(p) = g(p+\mathbf 1)g(p)=g(p+1).

Significance

The result itself. Discrete midpoint convexity (Theorem 7.7) is philosophically important: it shows the lattice-submodularity definition of L-convexity is not an arbitrary discretization choice but coincides exactly with the most direct discrete analogue of ordinary midpoint convexity, the classical characterization of convex functions via f((p+q)/2)≤(f(p)+f(q))/2f((p+q)/2) \le (f(p)+f(q))/2f((p+q)/2)≤(f(p)+f(q))/2. The L-optimality criterion and L-proximity theorem give L-convex minimization the same algorithmic footing as M-convex minimization (chunk 06): scaling algorithms for L-convex objectives — which arise naturally from network flow and submodular-function duality — inherit a provable, dimension-and-scale-explicit distance guarantee between a coarse-scale local optimum and the true minimizer.

Formalizing it. No matching item exists on the platform for L-convex functions, discrete midpoint convexity, or the L-optimality/proximity theorems. This mission gives the first formal statement of these results, completing (alongside chunk 06's M-convex-function results) both halves of the exchange-axiom-based theory that chapter 8's conjugacy duality later unifies.

Difficulty

A natural shortcut, given the structural parallel to chunk 06, is to assume the L-proximity theorem's proof is a mechanical relabeling of the M-proximity theorem's proof. It is not: the M-convex proof (chunk 06) crucially uses the exchange axiom's additive four-term inequality to build a chain of strictly improving points, whereas the L-convex proof instead exploits (TRF[Z])'s periodicity directly — it reduces to the case pα=0p_\alpha = 0pα​=0 using translation invariance, then constructs a minimal (with respect to the lattice order) point among all sufficiently good solutions and shows this minimality, combined with submodularity (SBF[Z]), forces the componentwise bound. The vector (rather than norm) form of the conclusion is not cosmetic: it is exactly what this lattice-order argument naturally produces, and is the form needed by later chapters' applications.

Formalization scope

The ground set VVV is a Fintype with DecidableEq; g:ZV→R∪{+∞}g : \mathbb Z^V \to \mathbb R \cup \{+\infty\}g:ZV→R∪{+∞} is (V → ℤ) → WithTop ℝ. Unlike chunk 06's M-convex axiom, (SBF[Z]), (TRF[Z]), and (SBF♮^\natural♮[Z]) are stated for all of ZV\mathbb Z^VZV, not restricted to dom⁡g\operatorname{dom} gdomg, so no explicit import of chunk 05's L-convex-set vocabulary was needed for dom g's structure (unlike the corresponding note in chunk 06's BRIEF.md, which flagged the same concern for dom f). L♮^\natural♮-convexity is represented via an explicit lift to Option V, matching the book's own primary definition, with the direct axiom (SBF♮^\natural♮[Z]) kept as a separate object related to it by Theorem 7.1.

A trivializing formalization of the goal would convert its componentwise vector bound into an ℓ∞\ell^\inftyℓ∞-norm bound (losing the direction-of-approach information the vector form carries) or drop Part (1)'s periodicity hypothesis g(p)=g(p+1)g(p) = g(p+\mathbf 1)g(p)=g(p+1); neither is done here. Propositions establishing dom g as an L-convex set, the L/L♮^\natural♮ relationship (Theorem 7.3), the submodular-set-function embedding (Proposition 7.4), and several structural closure properties are cut from this mission's scope (see MODERATION_NOTES.md) but are natural targets for a follow-on mission or for chunk 09, which builds directly on this chunk's exchange-axiom vocabulary, mirroring chunks 06→07.

Selected references

  • K. Murota, Discrete Convex Analysis, SIAM, 2003. DOI: 10.1137/1.9780898718508.
18 thms2 active usersReviewed
Convex OptimizationFunctional Analysis·Captain: mikedeng1

On the Maximal Monotonicity of Subdifferential Mappings II: Subdifferentials Are Exactly the Maximal Cyclically Monotone Operators, Unique up to an Additive ConstantResearch Paper

Motivation

A differentiable convex function on Rn\mathbb{R}^nRn is determined, up to an additive constant, by its gradient, and a vector field is a gradient of a convex function exactly when it satisfies a monotonicity condition along closed cycles. Convex analysis and optimization need the same statement for nonsmooth and extended-valued functions on infinite-dimensional spaces: the subdifferential replaces the gradient, and the question becomes which multivalued maps from a Banach space to its dual arise as subdifferentials, and how much of the function they determine. The answer underlies the treatment of optimality conditions, variational inequalities and evolution equations governed by subdifferentials, where one works with the operator ∂f\partial f∂f and needs to recover fff from it.

Timeline.

  • 1966: R. T. Rockafellar, Characterization of the subdifferentials of convex functions, Pacific J. Math. 17 (DOI 10.2140/pjm.1966.17.497), studied cyclically monotone operators and stated the characterization of subdifferentials as the maximal cyclically monotone operators (its Theorem 3), together with the maximal monotonicity of subdifferentials (its Theorem 4).
  • 1969: H. Brézis pointed out a gap in the 1966 proofs of maximality and uniqueness: a family of dual vectors xε∗x_\varepsilon^*xε∗​ used in the argument might increase unboundedly in norm as ε→0\varepsilon \to 0ε→0.
  • 1970: Rockafellar, On the maximal monotonicity of subdifferential mappings, Pacific J. Math. 33 (DOI 10.2140/pjm.1970.33.209), repaired the argument for arbitrary real Banach spaces, proving Theorem A (maximal monotonicity of ∂f\partial f∂f) and Theorem B (the characterization treated here).

Setting

Let EEE be a real Banach space with dual E∗E^*E∗, and write ⟨x,x∗⟩\langle x, x^* \rangle⟨x,x∗⟩ for the value of x∗∈E∗x^* \in E^*x∗∈E∗ at x∈Ex \in Ex∈E. A proper convex function on EEE is a function f:E→(−∞,+∞]f : E \to (-\infty, +\infty]f:E→(−∞,+∞], not identically +∞+\infty+∞, with f((1−λ)x+λy)≤(1−λ)f(x)+λf(y)f((1-\lambda)x + \lambda y) \le (1-\lambda)f(x) + \lambda f(y)f((1−λ)x+λy)≤(1−λ)f(x)+λf(y) for all x,y∈Ex, y \in Ex,y∈E and 0<λ<10 < \lambda < 10<λ<1. It is lower semicontinuous (lsc) in the norm topology. Its subdifferential is the multivalued map

∂f(x)={ x∗∈E∗∣f(y)≥f(x)+⟨y−x,x∗⟩  ∀y∈E }.\partial f(x) = \{\, x^* \in E^* \mid f(y) \ge f(x) + \langle y - x, x^* \rangle \ \ \forall y \in E \,\}.∂f(x)={x∗∈E∗∣f(y)≥f(x)+⟨y−x,x∗⟩  ∀y∈E}.

A multivalued map T:E→E∗T : E \to E^*T:E→E∗ is a cyclically monotone operator if

⟨x0−x1,x0∗⟩+⋯+⟨xn−1−xn,xn−1∗⟩+⟨xn−x0,xn∗⟩≥0whenever xi∗∈T(xi), i=0,…,n,\langle x_0 - x_1, x_0^* \rangle + \cdots + \langle x_{n-1} - x_n, x_{n-1}^* \rangle + \langle x_n - x_0, x_n^* \rangle \ge 0 \qquad\text{whenever } x_i^* \in T(x_i),\ i = 0, \dots, n,⟨x0​−x1​,x0∗​⟩+⋯+⟨xn−1​−xn​,xn−1∗​⟩+⟨xn​−x0​,xn∗​⟩≥0whenever xi∗​∈T(xi​), i=0,…,n,

and maximal cyclically monotone if, in addition, its graph {(x,x∗)∣x∗∈T(x)}\{(x, x^*) \mid x^* \in T(x)\}{(x,x∗)∣x∗∈T(x)} is not properly contained in the graph of any other cyclically monotone operator. The conjugate of fff is f∗(x∗)=sup⁡x∈E(⟨x,x∗⟩−f(x))f^*(x^*) = \sup_{x \in E} (\langle x, x^* \rangle - f(x))f∗(x∗)=supx∈E​(⟨x,x∗⟩−f(x)) on E∗E^*E∗, and j(x)=12∥x∥2j(x) = \tfrac12\|x\|^2j(x)=21​∥x∥2. In the Lean development these are ProperConvex, subdiff, IsCyclicallyMonotone, IsMaximalCyclicallyMonotone, conj and halfSqNorm, in the namespace RockafellarMaxMono.Cyclic.

Formalization targets

Goal: Theorem B (p. 210)

For every multivalued map T:E→E∗T : E \to E^*T:E→E∗ on a real Banach space EEE,

(∃f lsc proper convex with T=∂f)  ⟺  T is maximal cyclically monotone,\bigl(\exists f \text{ lsc proper convex with } T = \partial f\bigr) \iff T \text{ is maximal cyclically monotone},(∃f lsc proper convex with T=∂f)⟺T is maximal cyclically monotone,

and if fff and ggg are lsc proper convex with ∂f=T=∂g\partial f = T = \partial g∂f=T=∂g, then g=f+cg = f + cg=f+c for a real constant ccc. Both halves are one statement.

Milestones, in attack order

  1. (3.7) For a finite continuous convex function fff on a real Banach space, ∂f(x)\partial f(x)∂f(x) is nonempty and weak* compact and f′(x;u)=max⁡{⟨u,x∗⟩∣x∗∈∂f(x)}f'(x;u) = \max\{\langle u, x^* \rangle \mid x^* \in \partial f(x)\}f′(x;u)=max{⟨u,x∗⟩∣x∗∈∂f(x)}.
  2. Finite continuous case (pp. 214–215). For finite continuous convex f,gf, gf,g on a real Banach space, ∂g(x)⊃∂f(x)\partial g(x) \supset \partial f(x)∂g(x)⊃∂f(x) for all xxx implies g=f+constg = f + \mathrm{const}g=f+const.
  3. (3.1) ∂(f+j)(x)=∂f(x)+∂j(x)\partial(f + j)(x) = \partial f(x) + \partial j(x)∂(f+j)(x)=∂f(x)+∂j(x) for lsc proper convex fff.
  4. Proposition 1 x∗∗∈∂f∗(x∗)x^{**} \in \partial f^*(x^*)x∗∗∈∂f∗(x∗) if and only if x∗∗x^{**}x∗∗ is a weak** limit of a bounded net xix_ixi​ with xi∗∈∂f(xi)x_i^* \in \partial f(x_i)xi∗​∈∂f(xi​), xi∗→x∗x_i^* \to x^*xi∗​→x∗ in norm.
  5. (p. 213) (f+j)∗(f + j)^*(f+j)∗ is finite and continuous on E∗E^*E∗.
  6. (p. 211) f∗∗f^{**}f∗∗ restricted to EEE is fff.
  7. (3.6) For lsc proper convex f,gf, gf,g: ∂g(x)⊃∂f(x)\partial g(x) \supset \partial f(x)∂g(x)⊃∂f(x) for all xxx implies g=f+constg = f + \mathrm{const}g=f+const.

Significance

The result. Theorem B gives an intrinsic description of subdifferential maps: an operator is the subdifferential of a closed proper convex function if and only if it satisfies the cycle inequality and cannot be enlarged without violating it. The uniqueness clause says that a closed convex function is recovered from its subdifferential up to a constant, the nonsmooth counterpart of recovering a function from its gradient. Milestone 7 is stronger than uniqueness: a one-sided inclusion ∂f⊆∂g\partial f \subseteq \partial g∂f⊆∂g already forces g=f+cg = f + cg=f+c, and this is what gives maximality.

Formalizing it. The theorem is proved in the paper, and nothing of it is formalized on the platform. Mathlib has convex functions, continuous duals, biduals and weak-* topologies, but not extended-valued subdifferentials on Banach spaces, conjugate duality in the nonreflexive setting, or monotone operator theory. This mission produces formal statements of the paper's steps, the standard max formula for directional derivatives on a Banach space, and the Fenchel conjugate facts the argument uses, each as a separate target.

Difficulty

In a reflexive space the argument is short, because ∂f∗\partial f^*∂f∗ is the inverse of ∂f\partial f∂f. In a nonreflexive space it is not: ∂f∗\partial f^*∂f∗ maps E∗E^*E∗ into E∗∗E^{**}E∗∗, and points of E∗∗∖EE^{**} \setminus EE∗∗∖E appear. The naive route, transferring the inclusion ∂f⊆∂g\partial f \subseteq \partial g∂f⊆∂g to the conjugates by inverting the maps, breaks down there, and the 1966 argument failed at a related step, where the dual vectors in an approximation could be unbounded. Relating ∂f∗\partial f^*∂f∗ to ∂f\partial f∂f without reflexivity is where the difficulty sits; the boundedness of the approximating nets in Proposition 1 is essential and cannot be dropped. The finite continuous case and the max formula (3.7) are needed on an arbitrary Banach space, including the dual E∗E^*E∗, not only on EEE.

Formalization scope

EEE is a real Banach space (NormedAddCommGroup, NormedSpace ℝ, CompleteSpace); E∗E^*E∗ is StrongDual ℝ E with the operator norm, E∗∗E^{**}E∗∗ is StrongDual ℝ (StrongDual ℝ E), and E↪E∗∗E \hookrightarrow E^{**}E↪E∗∗ is NormedSpace.inclusionInDoubleDual. No reflexivity, inner product or finite dimension is assumed. Explicit readings:

  • Values in (−∞,+∞](-\infty, +\infty](−∞,+∞] are EReal with the requirement f(x)≠−∞f(x) \ne -\inftyf(x)=−∞; convexity is the paper's inequality for 0<λ<10 < \lambda < 10<λ<1 in EReal arithmetic. Lower semicontinuity is in the norm topology.
  • A multivalued map is E → Set (StrongDual ℝ E); T=∂fT = \partial fT=∂f means T(x)=∂f(x)T(x) = \partial f(x)T(x)=∂f(x) for every xxx.
  • The cycle inequality quantifies over all n∈Nn \in \mathbb{N}n∈N and points indexed by Fin (n + 1) with wrap-around addition, so the last term is ⟨xn−x0,xn∗⟩\langle x_n - x_0, x_n^* \rangle⟨xn​−x0​,xn∗​⟩. Maximality is graph inclusion among cyclically monotone operators, not among monotone operators.
  • The uniqueness constant is a real number, never ±∞\pm\infty±∞.
  • "⊃\supset⊃" in (3.6) is non-strict inclusion, and the hypothesis is one-sided.
  • A net is a nonempty directed partially ordered index type with convergence along atTop; weak** convergence is pointwise convergence on E∗E^*E∗; "bounded" is a uniform norm bound.
  • "Finite and continuous" for (f+j)∗(f+j)^*(f+j)∗ means equal everywhere to a continuous real-valued function. The max in (3.7) is IsGreatest, so it is attained; the directional derivative is the limit along λ→0+\lambda \to 0^+λ→0+.
  • The print's "∂(f+j)=∂f(x)+∂j(x)\partial(f + j) = \partial f(x) + \partial j(x)∂(f+j)=∂f(x)+∂j(x)" in (3.1) is read as ∂(f+j)(x)\partial(f+j)(x)∂(f+j)(x).

A formalization that drops lower semicontinuity, allows an extended-real constant, or replaces "maximal cyclically monotone" by "maximal monotone" states a different, and in the first two cases false or trivial, theorem; the statements here keep all three.

The proof reduces Theorem B to milestone 7 through Theorem 1 of Rockafellar (1966) and its Corollary 2, which are not stated in this paper and are not milestones; formal statements of them are welcome as supporting theorems. Contributions of reusable infrastructure are welcome: extended-valued subdifferentials and conjugates on normed spaces, the Fenchel–Moreau identity on EEE, the sum rule with a continuous function, and the max formula for directional derivatives.

Selected references

  • R. T. Rockafellar, On the maximal monotonicity of subdifferential mappings, Pacific J. Math. 33 (1970), 209–216. https://doi.org/10.2140/pjm.1970.33.209
  • R. T. Rockafellar, Characterization of the subdifferentials of convex functions, Pacific J. Math. 17 (1966), 497–510. https://doi.org/10.2140/pjm.1966.17.497
  • G. J. Minty, On the monotonicity of the gradient of a convex function, Pacific J. Math. 14 (1964), 243–247. https://doi.org/10.2140/pjm.1964.14.243
  • J.-J. Moreau, Fonctionnelles convexes, mimeographed lecture notes, Collège de France, 1967.
  • R. T. Rockafellar, Convex Analysis, Princeton University Press, 1970. https://doi.org/10.1515/9781400873173
13 thms2 active usersReviewed
🏆Completed
CombinatoricsConvex OptimizationDiscrete Geometry+1·Captain: Shuze Chen

Discrete Convex Analysis XXIII: Directional Derivatives and Subdifferentials of M-Convex FunctionsTextbook

Motivation

An M-convex function is defined on the integer lattice, but chapter 6's earlier results (companion missions 06-mconvex-functions-i, 22-ch06b-mconvexfunctions, 23-ch06c-mconvexfunctions) show it always extends to a genuine convex function on real space. Once that extension exists, every tool of classical convex analysis — directional derivatives, subdifferentials, positive homogeneity — becomes available, and the natural question is whether these classical objects remain combinatorially special when applied to an M-convex function's extension. This mission answers that question at its sharpest: the directional derivative of an M-convex function at any point is again a positively homogeneous M-convex function, its subdifferential is exactly the admissible-potential set of a distance function satisfying the triangle inequality, and this correspondence between positively homogeneous M-convex functions and triangle-inequality distance functions is itself a clean one-to-one correspondence. This closes the loop between chapters 4-5 (M-convex and L-convex sets, distance functions) and the continuous convex-analytic machinery chapter 8 needs for its duality theory.

Companion missions 06-mconvex-functions-i, 22-ch06b-mconvexfunctions, and 23-ch06c-mconvexfunctions cover this chapter's optimality theory, algebraic toolkit, and convex-extensibility characterization. This mission builds the vocabulary those results also need (redeclared here, since sibling drafts cannot yet import one another) and proves the chapter's real-variable capstones: the transfer of M-convexity's basic operations, optimality criterion, and supermodularity to the polyhedral (real-variable) setting, the identification of positively homogeneous M-convex functions with distance functions satisfying the triangle inequality, and — this mission's goal — the full directional-derivative/subdifferential correspondence.

Setting

Fix a finite ground set VVV. A polyhedral convex function g:RV→R∪{+∞}g : \mathbb R^V \to \mathbb R \cup \{+\infty\}g:RV→R∪{+∞} is (polyhedral) M-convex if it satisfies the real-variable exchange axiom (M-EXC[R]): for x,y∈dom⁡Rgx,y \in \operatorname{dom}_{\mathbb R} gx,y∈domR​g and u∈supp⁡+(x−y)u \in \operatorname{supp}^+(x-y)u∈supp+(x−y), some v∈supp⁡−(x−y)v \in \operatorname{supp}^-(x-y)v∈supp−(x−y) and α0>0\alpha_0 > 0α0​>0 make the exchange inequality hold on α∈[0,α0]\alpha \in [0,\alpha_0]α∈[0,α0​]; M♮-convex if its lift to one extra coordinate is M-convex. The directional derivative of ggg at x∈dom⁡Rgx \in \operatorname{dom}_{\mathbb R} gx∈domR​g in direction ddd is g′(x;d)=inf⁡t>0(g(x+td)−g(x))/tg'(x;d) = \inf_{t>0} (g(x+td) - g(x))/tg′(x;d)=inft>0​(g(x+td)−g(x))/t. A function is positively homogeneous if g(tx)=t⋅g(x)g(tx) = t \cdot g(x)g(tx)=t⋅g(x) for all t>0t > 0t>0; write 0M[R→R]0M[\mathbb R \to \mathbb R]0M[R→R] for the positively homogeneous polyhedral M-convex functions. A distance function γ\gammaγ satisfying the triangle inequality and its set of admissible potentials D(γ)D(\gamma)D(γ) were introduced in chapter 5; the subdifferential ∂Rf(x)={p:f(y)−f(x)≥⟨p,y−x⟩ ∀y}\partial_{\mathbb R} f(x) = \{p : f(y) - f(x) \ge \langle p, y-x \rangle\ \forall y\}∂R​f(x)={p:f(y)−f(x)≥⟨p,y−x⟩ ∀y} generalizes this to any function fff at a point xxx in its domain.

Formalization targets

Goal: the directional-derivative/subdifferential correspondence

For f∈M[R→R]f \in M[\mathbb R \to \mathbb R]f∈M[R→R] and x∈dom⁡Rfx \in \operatorname{dom}_{\mathbb R} fx∈domR​f, setting γf,x(u,v)=f′(x;−χu+χv)\gamma_{f,x}(u,v) = f'(x;-\chi_u+\chi_v)γf,x​(u,v)=f′(x;−χu​+χv​):

γf,x satisfies the triangle inequality,∂Rf(x)=D(γf,x)≠∅,f′(x;⋅)=γf,x^(⋅),\gamma_{f,x} \text{ satisfies the triangle inequality}, \quad \partial_{\mathbb R} f(x) = D(\gamma_{f,x}) \ne \emptyset, \quad f'(x;\cdot) = \widehat{\gamma_{f,x}}(\cdot),γf,x​ satisfies the triangle inequality,∂R​f(x)=D(γf,x​)=∅,f′(x;⋅)=γf,x​​(⋅),

with the analogous statement for f∈M[Z→R]f \in M[\mathbb Z \to \mathbb R]f∈M[Z→R] at an integer point xxx, using γf,x(u,v)=f(x−χu+χv)−f(x)\gamma_{f,x}(u,v) = f(x-\chi_u+\chi_v)-f(x)γf,x​(u,v)=f(x−χu​+χv​)−f(x) (Theorem 6.61). This is the weakest stable form: it identifies the subdifferential exactly, as a set, rather than bounding its size or complexity, and holds at every point of the domain uniformly.

Supporting structural targets

Ten further results build the real-variable toolkit and the positive-homogeneity correspondence this goal completes: the transfer of M♮-convexity, the basic operations, the optimality criterion, supermodularity, and weighted-minimizer polyhedrality to the real-variable setting (Theorems 6.48-6.52, Proposition 6.53), the identification of the classes 0M[Z∣R→R]0M[\mathbb Z|\mathbb R \to \mathbb R]0M[Z∣R→R] and 0M[R→R]0M[\mathbb R \to \mathbb R]0M[R→R] and the compatibility of convex extension with positive homogeneity (Proposition 6.56), the two directions of the correspondence between positively homogeneous M-convex functions and triangle-inequality distance functions (Propositions 6.57-6.58, Theorem 6.59), and the fact that a directional derivative of an M-convex function is itself positively homogeneous and M-convex (Proposition 6.60).

Significance

Theorem 6.61 is the technical bridge that lets discrete convex analysis borrow the entire apparatus of classical convex duality: because the subdifferential of an M-convex function is always the admissible-potential set of a chapter-5 distance function, every fact already proved about D(γ)D(\gamma)D(γ) (its polyhedral structure, its own L-convexity, its relationship to shortest paths) transfers immediately to subdifferentials of M-convex functions. This is exactly the mechanism the book calls out as essential for Chapter 8's separation theorem for M♮-convex functions. The 0M↔T0M \leftrightarrow T0M↔T correspondence (Theorem 6.59) is independently significant: it says the positively homogeneous special case of M-convex function theory — which is what directional derivatives of any M-convex function reduce to, by Proposition 6.60 — is exactly as rich as ordinary shortest-path distance function theory, no more and no less, so nothing new needs to be built to understand local behavior at a point.

None of these results are open — they are Murota's account of how the discrete exchange axiom interacts with directional differentiation and subgradients, a bridge chapter between the purely combinatorial theory of chapters 4-6 and the duality theory of chapter 8. What this mission contributes is a faithful, machine-checked formal statement of each, extending the shared Lean vocabulary (MExchangeAxiomR, DirDeriv, GammaHat) the Discrete Convex Analysis series builds on; no comparable formalization exists on the platform (see Formalization scope).

Difficulty

The naive approach to Theorem 6.61 would try to compute ∂Rf(x)\partial_{\mathbb R} f(x)∂R​f(x) directly from the definition of subgradient and separately verify it happens to equal some D(γ)D(\gamma)D(γ); the book's actual proof instead derives the equality of sets from the M-optimality criterion (Theorem 6.52) applied pointwise: p∈∂Rf(x)p \in \partial_{\mathbb R} f(x)p∈∂R​f(x) is shown, via a chain of logical equivalences, to be exactly the condition defining D(γf,x)D(\gamma_{f,x})D(γf,x​), so no separate verification of polyhedrality or nonemptiness is needed beyond what Theorem 6.52 and Proposition 6.60 already supply. The genuine difficulty is upstream, in Proposition 6.60 itself: showing a directional derivative is M-convex requires exploiting the local validity of the identity f(x+d)−f(x)=f′(x;d)f(x+d)-f(x) = f'(x;d)f(x+d)−f(x)=f′(x;d) for small ∥d∥1\|d\|_1∥d∥1​ (Eq. (6.85)) and then extending the exchange property from that neighborhood to all of RV\mathbb R^VRV using positive homogeneity — a two-step argument with no single-step shortcut, since the exchange axiom's defining inequality is not obviously homogeneous-invariant on its own.

Formalization scope

Ground-set elements are a Fintype V with DecidableEq; real-domain functions are (V→ℝ)→WithTop ℝ. The directional derivative is built directly as an infimum of difference quotients over t>0t>0t>0, matching the book's own local characterization (Eq. (6.85)) without a separate limit construction. Positive homogeneity and the classes 0M[R→R]/0M[Z→R] are stated exactly as the book defines them (the latter via positive homogeneity of the convex extension, not of f itself, since f is undefined off Zⱽ). Theorems 6.49-6.50 restate 4 of their 8 operations (matching the identical scope decision for chunk 22-ch06b-mconvexfunctions's Theorem 6.13); Theorem 6.61 omits the dual-integral refinement clauses for the M[R→R|Z]/ M[Z→Z] sub-classes. Both reductions are documented, not trivializing omissions — see Difficulty above and HARD.md/MODERATION_NOTES.md. No numeric constants are hard-coded anywhere in this mission. This mission's definitions are redeclared from chunks 06-mconvex-functions-i, 21-ch05b-lconvexsets (for the distance-function/admissible-potential vocabulary), 22-ch06b-mconvexfunctions, and 23-ch06c-mconvexfunctions rather than imported, since sibling drafts in this series cannot yet reference one another. Contributions completing any of the twelve sorrys are welcome; the goal and Proposition 6.60 carry the most independent proof content.

Selected references

  • K. Murota, Discrete Convex Analysis, SIAM, 2003. DOI: 10.1137/1.9780898718508.
  • K. Murota and A. Shioura, "M-convex function on generalized polymatroid," Mathematics of Operations Research, 24 (1999), pp. 95-105 (the polyhedral M-convex function theory this mission's real-variable results are drawn from).
56 thms2 active usersReviewed
🏆Completed
Dynamic ProgrammingOptimizationProbability·Captain: Shuze Chen

Markov Decision Processes XX: Perpetual American Options and Credit GrantingTextbook

Motivation

An American option may be exercised at any moment up to maturity, so pricing one is not an integration problem but a stopping problem: the holder must decide, at each date and in each state of the market, whether the payoff available now beats the option value of waiting. A perpetual American put pushes this to its limit — there is no maturity at all, so the horizon is unbounded and the problem has no terminal condition to induct backwards from. What replaces the terminal condition is a fixed point characterization, and the classical answer, going back to McKean (1965) and Merton (1973) in continuous time and to Cox, Ross and Rubinstein (1979) in the binomial model, is that the price is the smallest superharmonic majorant of the payoff.

Bäuerle and Rieder's Chapter 11 (Markov Decision Processes with Applications to Finance, Springer, 2011) derives this from their own general unbounded-horizon stopping theory rather than from stochastic analysis, and in the same chapter applies the bounded-horizon version to a problem from banking rather than trading: when should a bank cancel a credit line? The two halves share one mathematical shape — a stopping problem whose optimal policy turns out to be of threshold type — and this mission formalizes both, with the perpetual put as the goal.

Setting

The binomial model (§11.1). A stock moves from price xxx to xuxuxu with risk-neutral probability qqq and to xdxdxd with 1−q1-q1−q, where 0<d<u0 < d < u0<d<u, and the discount factor is β∈(0,1]\beta \in (0,1]β∈(0,1]. The defining relation of the risk-neutral measure,

βqu+β(1−q)d=1,\beta q u + \beta(1-q)d = 1,βqu+β(1−q)d=1,

is carried as a hypothesis of the model: it is what makes the discounted stock price a martingale, and the proofs use it directly.

The American put with strike KKK pays (K−x)+(K-x)^+(K−x)+ when exercised. With nnn periods to maturity its price satisfies the recursion J0(x)=(K−x)+J_0(x) = (K-x)^+J0​(x)=(K−x)+ and

Jn(x)=max⁡{(K−x)+, β(qJn−1(xu)+(1−q)Jn−1(xd))}=:TJn−1(x),J_n(x) = \max\Big\{(K-x)^+,\ \beta\big(q J_{n-1}(xu) + (1-q)J_{n-1}(xd)\big)\Big\} =: \mathcal{T}J_{n-1}(x),Jn​(x)=max{(K−x)+, β(qJn−1​(xu)+(1−q)Jn−1​(xd))}=:TJn−1​(x),

the maximum being "exercise now" against "hold". Proposition 11.1.2 describes the price πn(x):=JN−n(x)\pi_n(x) := J_{N-n}(x)πn​(x):=JN−n​(x) at time nnn of an option maturing at NNN: it is continuous in xxx, decreasing in nnn, and — the part that carries the argument — x↦πn(x)+xx \mapsto \pi_n(x) + xx↦πn​(x)+x is increasing, even though πn\pi_nπn​ itself decreases in xxx. That single reformulation, obtained by adding xxx to both sides of the recursion and using the risk-neutral relation, is what yields the threshold structure: there are exercise boundaries K=:xN∗≥xN−1∗≥⋯≥x0∗≥0K =: x_N^* \ge x_{N-1}^* \ge \dots \ge x_0^* \ge 0K=:xN∗​≥xN−1∗​≥⋯≥x0∗​≥0 with τ∗=inf⁡{n≤N∣Xn≤xn∗}\tau^* = \inf\{n \le N \mid X_n \le x_n^*\}τ∗=inf{n≤N∣Xn​≤xn∗​} optimal. Exercise when the stock falls far enough, and the boundary rises as maturity approaches.

The perpetual put (Theorem 11.1.3, the goal). With no expiration date the price at time zero is a supremum over all stopping times, τ≤∞\tau \le \inftyτ≤∞ included:

P(x):=sup⁡τ≤∞ExQ[βτ(K−Sτ)],P(x) := \sup_{\tau \le \infty} \mathbb{E}^{\mathbb{Q}}_x\big[\beta^\tau (K - S_\tau)\big],P(x):=τ≤∞sup​ExQ​[βτ(K−Sτ​)],

with the stopping reward set to zero on {τ=∞}\{\tau = \infty\}{τ=∞}. The theorem says four things: PPP is the limit of the finite-maturity prices JnJ_nJn​; PPP solves TP=P\mathcal{T}P = PTP=P and satisfies 0≤P≤K0 \le P \le K0≤P≤K; PPP is the smallest superharmonic function majorizing (K−x)+(K-x)^+(K−x)+; and, if the value Jf∗J_{f^*}Jf∗​ of the exercise-region policy dominates TJf∗\mathcal{T}J_{f^*}TJf∗​, then PPP equals that value and τ∗=inf⁡{n∣Xn∈E∗}\tau^* = \inf\{n \mid X_n \in E^*\}τ∗=inf{n∣Xn​∈E∗}, the hitting time of E∗={x∣P(x)=(K−x)+}E^* = \{x \mid P(x) = (K-x)^+\}E∗={x∣P(x)=(K−x)+}, is optimal.

The conditional in part d) is the book's own and is not decoration: without it the exercise region need not deliver an optimal stopping time, and the unconditional version is a different, false statement. The boundedness 0≤P≤K0 \le P \le K0≤P≤K in b) is likewise a genuine claim rather than a side remark — the fixed point equation alone admits other solutions, and it is boundedness together with minimality that pins PPP down among them.

Credit granting (§11.2). A bank holds a credit contract of maximal duration NNN. Each period it observes a rating class xnx_nxn​ evolving as a Markov process QXQ^XQX, and chooses to extend — earning c(x)c(x)c(x) — or to cancel, ending the contract. The value iteration is Jn(x)=max⁡{0, c(x)+β∫Jn−1 dQX(⋅∣x)}J_n(x) = \max\{0,\ c(x) + \beta\int J_{n-1}\,dQ^X(\cdot|x)\}Jn​(x)=max{0, c(x)+β∫Jn−1​dQX(⋅∣x)}. Under two structural assumptions — ccc increasing, and QXQ^XQX stochastically monotone, so that a better-rated borrower stays better-rated — Theorem 11.2.1 gives the same shape of answer as the option: cancel exactly when the rating falls below a threshold xn∗x_n^*xn∗​, and those thresholds rise as the remaining duration shortens, since a marginal borrower is no longer worth keeping when there is little time left to recover.

Theorem 11.2.2 repeats this when the borrower is not rated at all. The bank has only a prior μ0\mu_0μ0​ on the repayment probability and one signal per period; the state (s,n)(s,n)(s,n) records sss positive signals out of nnn, and the expected repayment probability is the posterior mean q(s,n)q(s,n)q(s,n). Monotonicity here is for the order of p. 342 — more positive signals and fewer negative ones — and not the coordinatewise order, under which the claim would be false: an extra signal that is negative makes the state worse.

What is being asked

Formalize Theorem 11.1.3 in full, all four parts: the limit identification, the fixed point equation with its bounds, the minimality among superharmonic majorants, and the conditional optimality of the exercise-region stopping time. The three milestones are Proposition 11.1.2, the finite-horizon put whose threshold structure the perpetual case specializes, and the two credit granting theorems, which run the same bounded-horizon argument on a different model.

The stopping-time apparatus is built rather than assumed: the stock path, the law Q\mathbb{Q}Q pinned by its finite-dimensional distributions, stopping times valued in N∪{∞}\mathbb{N}\cup\{\infty\}N∪{∞}, and the reward vanishing at ∞\infty∞. Part a) of the goal is the identification of the supremum with lim⁡nJn\lim_n J_nlimn​Jn​, so carrying PPP as an abstract function would make the theorem vacuous.

6 thms2 active usersReviewed
Dynamic ProgrammingOptimizationProbability·Captain: Shuze Chen

Markov Decision Processes XIX: Theory of Optimal Stopping ProblemsTextbook

Motivation

A gambler watching a sequence unfold has to decide, at each moment and knowing only the past, whether to take what is on the table or wait for something better. That is the whole of optimal stopping, and it is one of the few problems in stochastic control with a clean and completely general answer: the value of the problem is the smallest superharmonic function dominating the immediate payoff. Snell (1952) proved the martingale form; the dynamic-programming form is due to Chow, Robbins and Siegmund. It is the structure behind the pricing of American options, the secretary problem, sequential hypothesis testing, and the bandit problems of Chapter 5.

Bäuerle and Rieder's Chapter 10 (Markov Decision Processes with Applications to Finance, Springer, 2011) derives this from their own Markov-decision machinery rather than from martingale theory, which makes the whole development elementary and self-contained: a stopping problem is a Markov Decision Problem whose action space is {continue, stop}, so Chapter 2's finite-horizon theory and Chapter 7's unbounded-horizon theory apply to it verbatim. The chapter then runs the resulting theory on three classical problems and solves each one in closed form.

Setting

The problem. A Markov process (X_n) on a Borel space E is observed. A stopping time is a random time τ with {τ ≤ n} ∈ F_n — "upon observing the process until time n we can decide whether or not τ has already occurred". Stopping at τ collects

Rτ:=∑k=0τ−1ck(Xk)+gτ(Xτ),R_\tau := \sum_{k=0}^{\tau-1} c_k(X_k) + g_\tau(X_\tau),Rτ​:=k=0∑τ−1​ck​(Xk​)+gτ​(Xτ​),

a running reward c_k while continuing and a stopping reward g_τ at the end, and the problem is to find V_N^*(x) := sup_{τ ≤ N} E_x[R_τ] (10.1). Assumption (B_N) — finiteness of the supremum of the positive parts — is what makes this well posed.

The reduction (Theorem 10.1.2). Take A = {0,1}, let a = 0 mean continue and a = 1 mean stop, and make the transition law uncontrollable on continuation and absorbing on stopping. A policy π = (f_0,…,f_{N-1}) induces the stopping time τ_π = inf{n | f_n(X_n) = 1} ∧ N, and conversely every stopping time is a history-dependent policy. The theorem says the two suprema agree: the extra history buys nothing.

The recursion (Theorems 10.1.3, 10.1.5). The Bellman operator becomes a two-branch maximum,

Tv(x)=max⁡{g(x), c(x)+β∫v(x′)QX(dx′∣x)},\mathcal{T}v(x) = \max\Big\{g(x),\ c(x) + \beta\int v(x')Q^X(dx'|x)\Big\},Tv(x)=max{g(x), c(x)+β∫v(x′)QX(dx′∣x)},

with no action variable left in it. In the stationary case J_0 = g, J_n = \mathcal{T}J_{n-1}; the J_n increase, the sets S_n^* = {J_n = g} shrink — "the tendency to stop is non-decreasing as time goes by" — and the optimal rule is "stop on first entry into S_{N-n}^*".

The unbounded horizon (§10.2). Now the reward is discounted, R_τ = Σ β^k c(X_k) + β^τ g(X_τ) for τ < ∞, the value is V_∞^*(x) = sup_{τ<∞} E_x[R_τ], and there is no terminal condition to induct from. Three candidate values present themselves: V_∞^*; G = sup_π liminf_n J_{nπ}, a supremum over policies of limits of finite-horizon values; and J = lim_n J_n, which exists by monotonicity. Theorem 10.2.2, the goal, says all three coincide, that the common value solves J = \mathcal{T}J and satisfies 0-free bounds, and — the characterization — that it is the smallest c-superharmonic function majorizing g.

Turning the value into a rule (Theorems 10.2.3, 10.2.7, Corollaries 10.2.6, 10.2.8). Knowing the value is not knowing when to stop. Theorem 10.2.3 produces the stopping region as S^* = {J = g} = {d ≥ 0} where d = lim_n d_n, under two conditions that Corollary 10.2.6 then gives three checkable sufficient conditions for. Theorem 10.2.7 is the practical one, the One-Step-Look-Ahead Rule: if the set where stopping now beats stopping one step later is closed under the transition law, then the myopic rule is globally optimal. Corollary 10.2.8 adds monotonicity and gets a threshold.

Three applications (§10.3). The house seller who receives i.i.d. offers and pays maintenance on each rejection should accept the first offer above an explicit threshold, obtained as the maximiser of a one-dimensional function (Theorem 10.3.1). The secretary problem's value function is computed exactly (Proposition 10.3.2), giving the classical rule — reject the first k^*, then take the first leader — with success probability (k^*/N)h(k^*) and k^*(N)/N → 1/e (Theorem 10.3.3). And when the offers' distribution has an unknown parameter, MTP_2 of the likelihood propagates into monotonicity of the value in the information state (Theorem 10.3.4), with a fully explicit solution for the exponential/Inverse-Gamma conjugate pair (Theorem 10.3.6).

What is being asked

Formalize Theorem 10.2.2 in full: the three-way equality of V_∞^*, G and J, the fixed point equation, and — the part that carries the theorem — minimality among all functions that are both c-superharmonic and above g. Asserting only that J is such a function, or only one of the two conditions, is a strictly weaker and different claim.

The twelve milestones are the rest of the chapter, in attack order: the reduction and the two recursions, then the unbounded-horizon apparatus, then the three worked problems.

The stopping-time apparatus is built rather than assumed — the chain's law pinned by its finite-dimensional distributions, stopping times valued in ℕ ∪ {∞}, rewards vanishing at ∞ — because every theorem here is the identification of a supremum over stopping times with something computable, and carrying the value as an abstract function would make them vacuous. Every supremum is taken as a least upper bound against an explicit set of achievable values rather than by sSup, so that a set unbounded above is not silently given the value 0.

16 thms2 active usersReviewed
AnalysisStochastic Systems·Captain: mikedeng1

Diffusion approximations for open queueing networks with service interruptions 1: explicit Lipschitz bounds for the oblique reflection mapResearch Paper

Motivation

Heavy-traffic and fluid approximations for open queueing networks are obtained by writing the queue-content process as a deterministic function of a simpler netput process (arrivals minus potential service, corrected for routing) and then transferring a functional limit theorem for the netput through that function. The function is the multidimensional reflection map of Harrison and Reiman (Harrison and Reiman 1981), extended from continuous paths to paths with jumps by Reiman (Reiman 1984). The transfer works only if the map is continuous, and quantitative bounds on the approximation error require it to be Lipschitz with a known modulus.

Chen and Whitt (Chen and Whitt 1993) use this map to derive diffusion approximations for networks whose servers are subject to interruptions. Before doing so, Section 2 of the paper supplies "explicit Lipschitz bounds" for the map in the uniform topology: a bound in the Harrison–Reiman scaling (Proposition 2.1) and a new bound that depends on the routing matrix only through its powers (Proposition 2.3).

Timeline. Harrison and Reiman (1981) proved existence, uniqueness and continuity of the map on continuous paths for a routing matrix of spectral radius less than one. Reiman (1984) extended it to paths with jumps. Chen and Mandelbaum (Leontief systems, RBV's and RBM's, 1991, cited in the paper as [4]) noted that a minor extension of the argument makes the map Lipschitz on D([0,T],Rn)D([0,T],\mathbb R^n)D([0,T],Rn) with the uniform topology. Chen and Whitt (1993, Section 2) made the Lipschitz constants explicit.

Setting

Fix a dimension nnn and an n×nn\times nn×n matrix QQQ whose transpose QtQ^{\mathsf t}Qt is substochastic: all entries of QQQ are nonnegative and every column sum of QQQ is at most 111. Assume also Qk→0Q^k \to 0Qk→0 as k→∞k\to\inftyk→∞. With Markovian routing, QtQ^{\mathsf t}Qt is the routing matrix of an open network of nnn queues.

Vectors c∈Rnc\in\mathbb R^nc∈Rn carry the norm ∥c∥=∑j∣cj∣\|c\| = \sum_j |c_j|∥c∥=∑j​∣cj​∣, and matrices carry the maximum absolute column sum ∥P∥=max⁡j∑i∣Pij∣\|P\| = \max_j \sum_i |P_{ij}|∥P∥=maxj​∑i​∣Pij​∣ (Eq. (2.5)). D([0,T],Rn)D([0,T],\mathbb R^n)D([0,T],Rn) is the space of paths that are right-continuous with left limits on [0,T][0,T][0,T]. For a path xxx, ∣x∣∈Rn|x|\in\mathbb R^n∣x∣∈Rn is the vector of coordinatewise sup norms, ∣x∣j=sup⁡0≤t≤T∣xj(t)∣|x|_j = \sup_{0\le t\le T}|x_j(t)|∣x∣j​=sup0≤t≤T​∣xj​(t)∣, and ∥x∥=∥∣x∣∥=∑jsup⁡t∣xj(t)∣\|x\| = \big\||x|\big\| = \sum_j \sup_{t}|x_j(t)|∥x∥=​∣x∣​=∑j​supt​∣xj​(t)∣.

The reflection of x∈Dx \in Dx∈D is the pair (y,z)=(ψ(x),ϕ(x))(y,z) = (\psi(x),\phi(x))(y,z)=(ψ(x),ϕ(x)) with y∈Dy \in Dy∈D and

z=x+(I−Q) y≥0,yj nondecreasing, yj(0)=0,∫0Tzj(t) dyj(t)=0(1≤j≤n).z = x + (I-Q)\,y \ge 0, \qquad y_j \text{ nondecreasing},\ y_j(0) = 0, \qquad \int_0^T z_j(t)\,dy_j(t) = 0 \quad (1\le j\le n).z=x+(I−Q)y≥0,yj​ nondecreasing, yj​(0)=0,∫0T​zj​(t)dyj​(t)=0(1≤j≤n).

The last condition says that yjy_jyj​ increases only when zj=0z_j = 0zj​=0. In queueing terms, zzz is the vector of queue contents and yyy the cumulative idleness. The operator πx(y)=(Qy−x)↑∨0\pi_x(y) = (Qy - x)^{\uparrow}\vee 0πx​(y)=(Qy−x)↑∨0, where f↑(t)=sup⁡0≤s≤tf(s)f^{\uparrow}(t) = \sup_{0\le s\le t} f(s)f↑(t)=sup0≤s≤t​f(s) coordinatewise, has the reflection as its fixed point (Eq. (2.4)). Write γ=∥Qn∥\gamma = \|Q^n\|γ=∥Qn∥.

Formalization targets

Goal: Proposition 2.3

For all x1,x2∈Dx_1,x_2\in Dx1​,x2​∈D with reflections (ψ(xi),ϕ(xi))(\psi(x_i),\phi(x_i))(ψ(xi​),ϕ(xi​)),

∣ψ(x1)−ψ(x2)∣≤(I−Q)−1∣x1−x2∣componentwise,(2.9)|\psi(x_1)-\psi(x_2)| \le (I-Q)^{-1}|x_1-x_2| \quad\text{componentwise},\tag{2.9}∣ψ(x1​)−ψ(x2​)∣≤(I−Q)−1∣x1​−x2​∣componentwise,(2.9) ∥ψ(x1)−ψ(x2)∥≤∥(I−Q)−1∥ ∥x1−x2∥≤∑k=0∞∥Qk∥ ∥x1−x2∥≤n1−γ∥x1−x2∥,(2.10)\|\psi(x_1)-\psi(x_2)\| \le \|(I-Q)^{-1}\|\,\|x_1-x_2\| \le \sum_{k=0}^\infty \|Q^k\|\,\|x_1-x_2\| \le \frac{n}{1-\gamma}\|x_1-x_2\|,\tag{2.10}∥ψ(x1​)−ψ(x2​)∥≤∥(I−Q)−1∥∥x1​−x2​∥≤k=0∑∞​∥Qk∥∥x1​−x2​∥≤1−γn​∥x1​−x2​∥,(2.10) ∥ϕ(x1)−ϕ(x2)∥≤(1+∥I−Q∥ ∥(I−Q)−1∥)∥x1−x2∥≤(1+2n1−γ)∥x1−x2∥.(2.11)\|\phi(x_1)-\phi(x_2)\| \le \big(1+\|I-Q\|\,\|(I-Q)^{-1}\|\big)\|x_1-x_2\| \le \Big(1+\frac{2n}{1-\gamma}\Big)\|x_1-x_2\|.\tag{2.11}∥ϕ(x1​)−ϕ(x2​)∥≤(1+∥I−Q∥∥(I−Q)−1∥)∥x1​−x2​∥≤(1+1−γ2n​)∥x1​−x2​∥.(2.11)

The constants are those of the paper. The goal fixes nothing beyond the standing assumptions on QQQ.

Milestones

  1. Existence and uniqueness of the reflection for x∈Dx\in Dx∈D with x(0)≥0x(0)\ge0x(0)≥0 (Section 2, p. 337).
  2. Eq. (2.4): given (2.1)–(2.2), the complementarity condition (2.3) is equivalent to y=πx(y)y = \pi_x(y)y=πx​(y).
  3. γ=∥Qn∥<1\gamma = \|Q^n\| < 1γ=∥Qn∥<1 (p. 338).
  4. Proposition 2.2: ∥πxk(y1)−πxk(y2)∥≤∥Qk∣y1−y2∣∥≤∥y1−y2∥\|\pi_x^k(y_1)-\pi_x^k(y_2)\| \le \|Q^k|y_1-y_2|\| \le \|y_1-y_2\|∥πxk​(y1​)−πxk​(y2​)∥≤∥Qk∣y1​−y2​∣∥≤∥y1​−y2​∥ for k≥1k\ge1k≥1, the factor γ\gammaγ for k≥nk\ge nk≥n, and πxk(y1)→ψ(x)\pi_x^k(y_1)\to\psi(x)πxk​(y1​)→ψ(x).
  5. Proposition 2.1: for Q∗=Λ−1QΛQ^* = \Lambda^{-1}Q\LambdaQ∗=Λ−1QΛ with Λ\LambdaΛ diagonal and ∥Q∗∥=α<1\|Q^*\| = \alpha<1∥Q∗∥=α<1, the moduli ∥Λ∥∥Λ−1∥/(1−α)\|\Lambda\|\|\Lambda^{-1}\|/(1-\alpha)∥Λ∥∥Λ−1∥/(1−α) for ψ\psiψ and 1+∥I−Q∥∥Λ∥∥Λ−1∥/(1−α)1 + \|I-Q\|\|\Lambda\|\|\Lambda^{-1}\|/(1-\alpha)1+∥I−Q∥∥Λ∥∥Λ−1∥/(1−α) for ϕ\phiϕ.
  6. Remark (2.1): for n=1n=1n=1, Q=0Q=0Q=0 the bounds are attained.
  7. Remark (2.2): for two queues in series, (2.10) gives modulus 222, while (2.7) gives at best 444 (every modulus ≥4\ge 4≥4 is attained, 444 at z=1/2z = 1/2z=1/2).

Significance

Proposition 2.3 makes the queue-content and idleness processes of an open network Lipschitz functions of the netput, in the uniform norm, with a modulus computed from the routing matrix alone. Combined with the fact that Lipschitz continuity in the uniform topology passes to the Skorohod J1J_1J1​ and M1M_1M1​ topologies (Section 2 of the paper), it is what turns a functional central limit theorem for arrival and service processes into a heavy-traffic limit for the network. The paper uses it in exactly this way in Sections 3–4. Explicit moduli also yield rates: an error of order ε\varepsilonε in the netput produces an error of at most nε/(1−γ)n\varepsilon/(1-\gamma)nε/(1−γ) in the idleness process.

On the formal side, the results are proved in the paper, but neither the reflection map nor D([0,T],Rn)D([0,T],\mathbb R^n)D([0,T],Rn) has a machine-checked development in Mathlib or on this platform. The mission would provide a reusable definition of the oblique reflection map with a Lebesgue–Stieltjes complementarity condition, its fixed-point characterization, and certified Lipschitz constants, as a foundation for any later formal heavy-traffic limit.

Difficulty

The componentwise bound (2.9) is short once the fixed-point form of the map is available. The difficulty lies in the infrastructure beneath it. The fixed-point characterization (2.4) is a one-dimensional Skorokhod-problem argument carried out coordinatewise for paths with jumps, where the complementarity condition must be handled through Lebesgue–Stieltjes measures. A jump of yjy_jyj​ is allowed at a time where zj=0z_j = 0zj​=0 even if zjz_jzj​ was positive just before. Existence needs the iterates πxk(0)\pi_x^k(0)πxk​(0) to converge in DDD and the limit to satisfy (2.1)–(2.3). The explicit constants involve (I−Q)−1(I-Q)^{-1}(I−Q)−1, ∑k∥Qk∥\sum_k\|Q^k\|∑k​∥Qk∥ and γ=∥Qn∥<1\gamma = \|Q^n\|<1γ=∥Qn∥<1. The last inequality is a combinatorial fact about transient substochastic matrices. It does not follow from ∥Q∥≤1\|Q\|\le1∥Q∥≤1.

Formalization scope

Vectors are Fin n → ℝ, matrices Matrix (Fin n) (Fin n) ℝ, and ∥P∥\|P\|∥P∥ is the maximum absolute column sum. Paths are functions ℝ → Fin n → ℝ, of which only the restriction to [0,T][0,T][0,T] matters. Membership in D([0,T],Rn)D([0,T],\mathbb R^n)D([0,T],Rn) is the predicate IsCadlagOn T x: right-continuous on [0,T)[0,T)[0,T), left limits on (0,T](0,T](0,T], and (redundantly) bounded on [0,T][0,T][0,T]. The reflection is the predicate IsReflection Q T x y z. Every theorem is stated for all pairs satisfying it, so no choice function and no junk value are involved. Condition (2.3) is encoded as "the Lebesgue–Stieltjes measure dyjdy_jdyj​ of {t∈[0,T]:zj(t)>0}\{t\in[0,T]: z_j(t)>0\}{t∈[0,T]:zj​(t)>0} is zero". For z≥0z\ge0z≥0 this is equivalent to ∫0Tzj dyj=0\int_0^T z_j\,dy_j=0∫0T​zj​dyj​=0. πxk\pi_x^kπxk​ is Nat.iterate, (I−Q)−1(I-Q)^{-1}(I−Q)−1 is Mathlib's matrix inverse (invertible under the standing assumptions), and ∑k∥Qk∥\sum_k\|Q^k\|∑k​∥Qk∥ is a tsum stated together with its summability.

Corrections and conventions, each disclosed in the item concerned:

  • The norm (2.6). The page prints ∥x∥=sup⁡t∑j∣xj(t)∣\|x\| = \sup_t\sum_j|x_j(t)|∥x∥=supt​∑j​∣xj​(t)∣. Under that norm Propositions 2.1 and 2.3 are false for n≥2n\ge2n≥2. With Q=0Q=0Q=0, n=2n=2n=2, T=1T=1T=1, x1≡0x_1\equiv0x1​≡0 and x2=(−1[0.1,0.2),−1[0.3,0.4))x_2 = (-\mathbf 1_{[0.1,0.2)}, -\mathbf 1_{[0.3,0.4)})x2​=(−1[0.1,0.2)​,−1[0.3,0.4)​), one gets ∥x1−x2∥=1\|x_1-x_2\|=1∥x1​−x2​∥=1 but ψ(x2)=(1[0.1,1],1[0.3,1])\psi(x_2) = (\mathbf 1_{[0.1,1]},\mathbf 1_{[0.3,1]})ψ(x2​)=(1[0.1,1]​,1[0.3,1]​) has norm 222. The paper's proofs are valid for ∥x∥=∑jsup⁡t∣xj(t)∣\|x\| = \sum_j\sup_t|x_j(t)|∥x∥=∑j​supt​∣xj​(t)∣, which is used throughout. In dimension one the two norms coincide.
  • (2.8) prints ϕ(x1)−ϕ(x1)\phi(x_1)-\phi(x_1)ϕ(x1​)−ϕ(x1​). The formalization states ϕ(x1)−ϕ(x2)\phi(x_1)-\phi(x_2)ϕ(x1​)−ϕ(x2​).
  • (2.2)–(2.3) print the index range 1≤j≤J1\le j\le J1≤j≤J. The dimension is nnn.
  • x(0)≥0x(0)\ge0x(0)≥0 is added to the existence item. Conditions (2.1)–(2.2) force z(0)=x(0)z(0)=x(0)z(0)=x(0), so no reflection exists otherwise. The Lipschitz bounds are stated for all solution pairs and are vacuous exactly when some xi(0)x_i(0)xi​(0) has a negative coordinate.
  • Proposition 2.1 assumes only that Λ\LambdaΛ is diagonal with nonzero entries. All quantities depend on ∣Λ∣|\Lambda|∣Λ∣, so this covers the positive scaling of Harrison and Reiman.
  • Eq. (2.4) keeps the standing assumptions on QQQ as on the page, although the equivalence does not use them.

A trivializing formalization would read (2.3) through a Bochner integral, which is 000 for non-integrable integrands, or take suprema over unbounded families. The measure-zero encoding and the boundedness built into IsCadlagOn rule both out. A sorry-free check shows that Remark (2.1)'s jump example satisfies IsReflection.

Welcome contributions: a general API for càdlàg paths on [0,T][0,T][0,T] (boundedness, measurability, running suprema), the one-dimensional Skorokhod lemma for càdlàg paths, and the Neumann series for transient substochastic matrices. All of these are reusable beyond this mission.

Selected references

  • H. Chen and W. Whitt, Diffusion approximations for open queueing networks with service interruptions, Queueing Systems 13 (1993) 335–359. https://doi.org/10.1007/BF01149260
  • J. M. Harrison and M. I. Reiman, Reflected Brownian motion on an orthant, Annals of Probability 9 (1981) 302–308. https://doi.org/10.1214/aop/1176994428
  • M. I. Reiman, Open queueing networks in heavy traffic, Mathematics of Operations Research 9 (1984) 441–458. https://doi.org/10.1287/moor.9.3.441
  • H. Chen and A. Mandelbaum, Discrete flow networks: diffusion approximations and bottlenecks, Annals of Probability 19 (1991) 1463–1519. https://doi.org/10.1214/aop/1176990220
10 thms2 active usersReviewed
🏆Completed
CombinatoricsOptimizationTheoretical Computer Science·Captain: mikedeng1

Approximation Algorithms for Combinatorial Problems III: The Worst-Case Ratio of the Weighted Algorithm B2Research Paper

Motivation

MAXIMUM SATISFIABILITY asks for a truth assignment satisfying as many clauses of a given set as possible. It was among the first NP-hard optimization problems for which an approximation algorithm came with a proven worst-case guarantee. In Approximation Algorithms for Combinatorial Problems (J. Comput. System Sci. 9, 1974, doi:10.1016/S0022-0000(74)80044-9), David S. Johnson set up a framework for measuring such guarantees and analyzed two heuristics for the problem. The second of these, algorithm B2, weights each clause by 2−∣C∣2^{-|C|}2−∣C∣ and repeatedly sets a literal so that the heavier side is satisfied. On inputs whose clauses all have at least kkk literals, it satisfies at least a 1−2−k1 - 2^{-k}1−2−k fraction of the optimum.

B2 is the ancestor of a line of work on MAX-SAT approximation:

  • 1974. Johnson proves the 2k/(2k−1)2^k/(2^k-1)2k/(2k−1) bound for B2 on MS(k) and the (k+1)/k(k+1)/k(k+1)/k bound for the unweighted greedy algorithm B1.
  • 1994. Goemans and Williamson (SIAM J. Discrete Math. 7) present Johnson's algorithm as the derandomization, by conditional expectations, of the uniformly random assignment, and combine it with LP rounding to obtain a 3/43/43/4-approximation.
  • 1999. Chen, Friesen and Zheng (JCSS 58) show that B2 is a 2/32/32/3-approximation on general inputs, sharper than the 1/21/21/2 that Johnson's bound gives at k=1k = 1k=1.

Setting

A literal is a variable xix_ixi​ or its negation xˉi\bar x_ixˉi​. A clause is a finite set of literals. A truth assignment is a set TTT of literals containing no pair {xi,xˉi}\{x_i, \bar x_i\}{xi​,xˉi​}; it satisfies a clause CCC when C∩T≠∅C \cap T \ne \emptysetC∩T=∅. An input is a finite set SSS of clauses, and S∗S^*S∗ is the largest ∣S′∣|S'|∣S′∣ over subsets S′⊆SS' \subseteq SS′⊆S satisfied by one truth assignment. MS(k) is the restriction to inputs in which every clause contains at least kkk distinct literals.

Algorithm B2 keeps a set SUB of satisfied clauses, a set LEFT of unsettled clauses, the literals LIT still available, and a weight w(C)w(C)w(C) per clause, starting from w(C)=2−∣C∣w(C) = 2^{-|C|}w(C)=2−∣C∣, SUB=∅\mathrm{SUB} = \emptysetSUB=∅, LEFT=S\mathrm{LEFT} = SLEFT=S. While some literal of LIT occurs in a clause of LEFT, it picks any such literal yyy. Let YT be the clauses of LEFT containing yyy and YF those containing yˉ\bar yyˉ​. If ∑YTw≥∑YFw\sum_{\mathrm{YT}} w \ge \sum_{\mathrm{YF}} w∑YT​w≥∑YF​w, it makes yyy true, moves YT to SUB and doubles the weight of each clause of YF; otherwise it does the symmetric move with yˉ\bar yyˉ​. Finally it removes y,yˉy, \bar yy,yˉ​ from LIT. When no literal of LIT remains in LEFT it returns SUB.

The choice of yyy is free, so several outputs may be choosable on one input. Johnson measures the algorithm by its worst choosable output, through the ratio r(B2,S)=S∗/∣SUB∣r(B2, S) = S^*/|\mathrm{SUB}|r(B2,S)=S∗/∣SUB∣ and its maximum R[B2,MS(k)](n)R[B2, \mathrm{MS}(k)](n)R[B2,MS(k)](n) over inputs of size at most nnn.

Formalization targets

Goal: Theorem 3, with the equality range corrected

For every k≥1k \ge 1k≥1, every SSS in MS(k) and every SUB choosable by B2 on SSS,

(2k−1) S∗≤2k ∣SUB∣,(2^k - 1)\,S^* \le 2^k\,|\mathrm{SUB}|,(2k−1)S∗≤2k∣SUB∣,

and for every k≥2k \ge 2k≥2 there are SSS in MS(k) and a choosable SUB with ∣SUB∣>0|\mathrm{SUB}| > 0∣SUB∣>0 and

(2k−1) S∗=2k ∣SUB∣.(2^k - 1)\,S^* = 2^k\,|\mathrm{SUB}|.(2k−1)S∗=2k∣SUB∣.

The paper states equality "for all sufficiently large nnn" for every k≥1k \ge 1k≥1. At k=1k = 1k=1 this is false: the ratio 2 would require every clause to be a unit clause and all clauses to be jointly satisfiable, and on such inputs B2 satisfies everything. The goal therefore asserts tightness for k≥2k \ge 2k≥2. A separate item states that at k=1k = 1k=1 the ratio is never attained.

Milestones

  1. Initially, the total weight of LEFT is at most ∣S∣/2k|S|/2^k∣S∣/2k.
  2. No iteration increases the total weight of LEFT, so at halting it is still at most ∣S∣/2k|S|/2^k∣S∣/2k.
  3. At halting, every clause left in LEFT has weight exactly 111.
  4. At halting, ∣LEFT∣≤∣S∣/2k|\mathrm{LEFT}| \le |S|/2^k∣LEFT∣≤∣S∣/2k and ∣SUB∣≥∣S∣(1−2−k)|\mathrm{SUB}| \ge |S|(1 - 2^{-k})∣SUB∣≥∣S∣(1−2−k).
  5. The eight-clause instance for k=3k = 3k=3 has S∗=8S^* = 8S∗=8 and admits a choosable output of size 777.

Significance

The bound 1−2−k1 - 2^{-k}1−2−k is exactly the expected fraction of clauses a uniformly random assignment satisfies on MS(k). B2 attains it deterministically, and the proof bounds ∣SUB∣|\mathrm{SUB}|∣SUB∣ against ∣S∣|S|∣S∣ rather than against S∗S^*S∗, a feature the paper points out. For k=3k = 3k=3 the constant 8/78/78/7 was later shown by Håstad (2001) to be optimal among polynomial-time algorithms unless P = NP, so the guarantee of this 1974 algorithm is best possible for MAX-E3-SAT.

Formalizing the result produces a machine-checked potential-function argument for a nondeterministic algorithm: the invariant links each clause's weight to how many of its literals have been removed. The mission also records a correction to the printed statement at k=1k = 1k=1. The paper's proof is complete; to our knowledge no machine-checked version exists.

Difficulty

The obvious argument follows the counting proof for algorithm B1 and compares clauses saved with clauses wounded in each step. It fails here: B2 can wound more clauses than it saves in a step, and the bound holds only in the weighted sense. The weight of a clause is not a static quantity. It is 2−∣C∣2^{-|C|}2−∣C∣ times 222 to the number of its literals already discarded, and this invariant must be carried through every step, including clauses that contain both yyy and yˉ\bar yyˉ​. Tightness needs an explicit input and an explicit adversarial run that exploits the tie in Step 4, and then a proof that every assignment of the remaining variables kills exactly one clause.

Formalization scope

  • Literals and clauses. A literal is a pair (variable index in N\mathbb NN, sign). Clauses are Finsets of literals, and an input is a Finset of clauses, so there are no duplicate clauses. Truth assignments are partial and consistent. S∗S^*S∗ is a maximum over the finite, nonempty family of satisfiable subsets.
  • B2 as a relation. B2 is a nondeterministic run relation, and "choosable" means reachable by finitely many steps from the initial state and halting. Step 3 allows a literal of either sign. The tie in Step 4 goes to yyy, and the comparison and doublings use the weights before the update. Weights are rationals.
  • Size-free ratios. The size-dependent R[B2,MS(k)](n)R[B2, \mathrm{MS}(k)](n)R[B2,MS(k)](n) is replaced by statements about every input (upper bound) and one attaining input (tightness). The two forms are equivalent because RRR is a maximum over finitely many inputs and nondecreasing in nnn.
  • No division. All ratios are multiplied out, so no division by zero can make a bound vacuous.
  • Trivializations ruled out. A deterministic tie-break in Step 3 or 4, or tightness at a single fixed kkk, would be a weaker theorem and is not acceptable. So is stating only the upper bound.

The definitions (the MS layer, the run relation) can be reused by other MAX-SAT approximation results, and an identical MS layer appears in the companion mission on algorithm B1. Contributions welcome: proofs of the milestones, of the k≥2k \ge 2k≥2 tightness family, and of the k=1k = 1k=1 correction.

Selected references

  • D. S. Johnson, Approximation algorithms for combinatorial problems, J. Comput. System Sci. 9 (1974) 256–278. https://doi.org/10.1016/S0022-0000(74)80044-9
  • J. Chen, D. K. Friesen, H. Zheng, Tight bound on Johnson's algorithm for maximum satisfiability, J. Comput. System Sci. 58 (1999) 622–640. https://doi.org/10.1006/jcss.1998.1610
  • M. X. Goemans, D. P. Williamson, New 3/4-approximation algorithms for the maximum satisfiability problem, SIAM J. Discrete Math. 7 (1994) 656–666. https://doi.org/10.1137/S0895480192243516
  • J. Håstad, Some optimal inapproximability results, J. ACM 48 (2001) 798–859. https://doi.org/10.1145/502090.502098
9 thms2 active usersReviewed
🏆Completed
Dynamic ProgrammingOptimizationProbability·Captain: Shuze Chen

Markov Decision Processes XVII: Random-Horizon Consumption-Investment and the De Finetti Dividend ProblemTextbook

Motivation

An insurance company collects premia and pays claims each period; the difference is a random, signed quantity that can push the company's risk reserve up or down. At the start of every period, before that period's premia and claims are realized, the company's owners may pay themselves a dividend out of the current reserve — but once the reserve goes negative the company is ruined and stops operating for good. How should the owners time and size these payments to maximize the total expected discounted dividend paid out before ruin? This is the classical De Finetti dividend problem, one of risk theory's oldest optimization questions, and Chapter 9 §9.2 of Bäuerle and Rieder's Markov Decision Processes with Applications to Finance (Springer, 2011) solves its fully discrete-time version by identifying the exact combinatorial shape of the optimal policy — not just proving one exists. This mission also covers §9.1, a different application of Chapter 7's contracting theory to a consumption-investment problem whose planning horizon is itself random rather than fixed or infinite.

Setting

The dividend model is a stationary Markov Decision Model on the integers: the state x∈Zx \in \mathbb Zx∈Z is the current risk reserve, the action a∈{0,1,…,x}a \in \{0,1,\dots,x\}a∈{0,1,…,x} (for x≥0x \ge 0x≥0; only a=0a=0a=0 is available once ruined) is the dividend paid, the reward is r(x,a):=ar(x,a):=ar(x,a):=a, and the reserve evolves by i.i.d. increments ZnZ_nZn​ (premia minus claims) after the dividend is deducted. Because the reward is bounded by an explicit function of the state (Lemma 9.2.2), Chapter 7's general existence theory applies directly, and the value function J∞J_\inftyJ∞​ satisfies a genuine Bellman equation. The chapter's real content begins once existence is established: Theorem 9.2.3 pins down enough analytic structure of J∞J_\inftyJ∞​ and its largest-maximizing policy f∗f^*f∗ (monotonicity, a Lipschitz-type inequality, and a self-consistency identity) to drive a purely combinatorial argument that f∗f^*f∗'s shape is a finite alternation of "pay nothing" and "pay down to a fixed level" intervals — a band-policy (Definition 9.2.5). Section 9.1's random-horizon consumption-investment model reuses the same Chapter 7 machinery in a different setting: the usual (c,a)(c,a)(c,a) (consumption, portfolio) decision each period, but where the horizon itself ends after each period with probability 1−p1-p1−p, making the effective one-period discount βp\beta pβp rather than β\betaβ.

Formalization targets

The goal, Theorem 9.2.9, states the section's main claim in one sentence: the stationary policy (f∗,f∗,… )(f^*,f^*,\dots)(f∗,f∗,…) is optimal and is a band-policy. Short as it is stated, its proof assembles every earlier result of the section. The milestones supply that assembly, in order: Lemma 9.2.2 gives the model's bounding function and the resulting integrability/convergence facts; Theorem 9.2.3 gives the value-function bounds and the self-consistency identity f∗(x−f∗(x))=0f^*(x-f^*(x))=0f∗(x−f∗(x))=0; Corollary 9.2.4 checks the two sign-definite degenerate cases directly from Theorem 9.2.3; Proposition 9.2.6 proves the top threshold ξ:=sup⁡{x∣f∗(x)=0}\xi := \sup\{x \mid f^*(x)=0\}ξ:=sup{x∣f∗(x)=0} is finite (not merely well-defined) and that f∗f^*f∗ is a simple barrier above it; Proposition 9.2.8 proves the increment property below ξ\xiξ that forces each band's shape; and Theorem 9.2.10 (a postscript refinement, stated after the goal) shows the wave lengths are bounded once the reserve's downward jumps are themselves bounded, collapsing to a single barrier-policy in the extreme case. Theorem 9.1.1, the random-horizon consumption-investment verification theorem, is included as a full item but is not a milestone of this goal, since its content and proof belong to a different, disjoint model — see Difficulty.

Significance

Band-policies and the discrete-time De Finetti dividend problem have no substrate anywhere in Mathlib or on the platform, and the result is a genuinely deep, classical one: a discrete-time analogue of the continuous-time De Finetti barrier-strategy theory, obtained here by pure dynamic-programming argument rather than the stochastic-calculus techniques the continuous-time theory usually relies on. The mission is explicit that the goal's conclusion is the general band-policy structure, not the strictly weaker barrier-policy special case that Theorem 9.2.10 b) proves only under an extra hypothesis (bounded downward jumps) — stating the goal with a barrier-policy conclusion instead would understate what Theorem 9.2.9 actually proves.

Difficulty

The central formalization challenge is Definition 9.2.5's own combinatorial intricacy: a band-policy is specified by an alternating chain of thresholds 0≤c0<d1≤c1<d2≤⋯≤dn≤cn0 \le c_0 < d_1 \le c_1 < d_2 \le \dots \le d_n \le c_n0≤c0​<d1​≤c1​<d2​≤⋯≤dn​≤cn​ with a positive-width gap condition on every wave, and the policy's four piecewise branches case-split on which wave (if any) the current state falls into. This mission renders it existentially over the witnessing (n,c,d)(n,c,d)(n,c,d) rather than as one closed-form function, a faithful but more verbose transcription that avoids conflating the different branch conditions. A second difficulty is Proposition 9.2.6's own finiteness claim: ξ\xiξ is a supremum over a subset of N0\mathbb N_0N0​ that could, in principle, be unbounded, and Mathlib's convention for sSup over the naturals returns a finite junk value (000) even for an unbounded set — using it directly would silently trivialize "ξ<∞\xi<\inftyξ<∞" into a claim that is true regardless of the proposition's actual mathematical content. This mission instead states the proposition by exhibiting the finite value of ξ\xiξ directly, so that "ξ\xiξ is finite" survives as genuine content that the theorem's proof must establish. A third difficulty is scope: Theorem 9.1.1's random-horizon consumption-investment model shares no state space, action space, or definitions with the dividend model of the goal, despite both appearing in this chunk's assigned page range; it is formalized as a genuine application of a locally-restated copy of Chapter 7's contracting theory, but is excluded from the milestone list proper since it plays no role in the goal's own proof.

Formalization scope

The dividend model's transition law is built from Mathlib's PMF (probability mass function) type on Z\mathbb ZZ, which supplies the "probabilities sum to one" fact automatically rather than as a separate hypothesis. J_\infty, \delta, and every finite-horizon value function throughout this mission use this whole book series' Filter.limsup-of-truncations convention for infinite-horizon reward, restated locally (own namespace copy, per this series' file-ownership boundary) from chunk 07a's identical apparatus rather than imported. The consumption-investment model of §9.1 is formalized with the number of risky assets ddd as an explicit type parameter and its admissible-portfolio and domain restrictions as separate, citable fields rather than folded silently into the reward or transition definitions.

Selected references

  • N. Bäuerle and U. Rieder, Markov Decision Processes with Applications to Finance, Universitext, Springer, 2011. DOI: 10.1007/978-3-642-18324-9.
  • B. De Finetti, "Su un'impostazione alternativa della teoria collettiva del rischio", Transactions of the XVth International Congress of Actuaries, 1957 (the original continuous-time dividend problem this chapter's discrete-time analogue is modeled on).
  • H. Schmidli, Stochastic Control in Insurance, Springer, 2008 (cited by Remark 9.2.1 for the reduction from a continuous dividend-payout action space to the integer setting used throughout this section).
  • H. U. Gerber, "Games of economic survival with discrete- and continuous-income processes", Operations Research, 1972 (an early discrete-time treatment of the same class of problems, in the spirit this chapter's own model follows).
11 thms2 active usersReviewed
🏆Completed
Dynamic ProgrammingOptimizationProbability·Captain: Shuze Chen

Markov Decision Processes XVI: Piecewise Deterministic Markov Decision ProcessesTextbook

Motivation

Every mission in this series so far has treated a control problem that already lives in discrete time: a decision maker observes a state, chooses an action, and the process moves to a new state at the next integer time step. Many real systems evolve in continuous time instead — a machine that runs deterministically until it randomly breaks down and is repaired into a new condition, an inventory that drains continuously until a random demand arrives, a population that grows deterministically between random catastrophic events. Chapter 8 of Bäuerle and Rieder's Markov Decision Processes with Applications to Finance (Springer, 2011) shows that an entire class of such continuous-time control problems — Piecewise Deterministic Markov Decision Processes, where the state moves along a deterministic, controlled flow between randomly-timed jumps to a new state — can be solved by exactly the discrete-time machinery this book's series has already built, once the problem is re-expressed as a Markov Decision Model at the jump times themselves. This mission covers that embedding and its consequences (§8.2), and a simpler, discrete-state special case, the continuous-time Markov Decision Chain, treated with both an infinite and a finite time horizon (§8.3).

Setting

A Piecewise Deterministic Markov Decision Model (Definition 8.1.1) consists of a Borel state space EEE, a Borel control space UUU, a deterministic drift μ(x,u)\mu(x,u)μ(x,u) governing the flow φtα(x)\varphi^\alpha_t(x)φtα​(x) between jumps, a Poisson jump clock of rate λ\lambdaλ, a kernel QQQ giving the distribution of the post-jump state, a reward rate rrr, and a discount rate β\betaβ. A control is a whole measurable function α:R≥0→U\alpha : \mathbb R_{\ge0} \to Uα:R≥0​→U fixed at each jump time and applied until the next one — so the "action space" of the embedded discrete-time problem is itself a function space, a genuinely new technical wrinkle this book's earlier chapters never face. Embedding at the jump times produces a discrete-time Markov Decision Model (E,A,Q′,r′)(E,A,Q',r')(E,A,Q′,r′) whose reward and kernel are themselves integrals of the original data against the flow (Eqs. (8.4)-(8.5)); Chapter 7's infinite-horizon existence theory, already developed for a general Borel state space, applies directly to this embedded model once its own compactness and semicontinuity hypotheses are checked. Checking them forces a further enlargement of the control space to the relaxed controls RRR — measurable functions into probability measures on UUU rather than UUU itself — which is compact in a suitable topology where the space of literal control functions is not.

Formalization targets

The goal, Theorem 8.2.6, is the chapter's central existence result: given a continuous upper bounding function with the discrete embedded model's own contraction-type condition and a package of continuity/compactness assumptions, the value function of the model embedded with relaxed controls is bounded, upper semicontinuous, and a genuine fixed point of the maximal- reward operator, and an optimal relaxed Markov policy exists. The milestones build up to it and extend past it: Theorem 8.2.1 establishes the foundational fact that the continuous-time expected reward of the original process equals the discrete-time embedded model's own value — the correspondence every other result in the chapter relies on; Lemma 8.2.5 is the technical semicontinuity-preservation step the goal's proof needs; Theorem 8.2.7 upgrades the goal's relaxed optimal policy to a genuine, nonrelaxed one under an uncontrolled-flow or convexity condition; Theorem 8.2.8 gives the classical Hamilton-Jacobi-Bellman verification technique as an alternative, differential route to the same value function. Section 8.3's continuous-time Markov Decision Chain — the same theory specialized to a countable state space, transition rates in place of a kernel, and an uncontrolled flow — is covered for both an infinite horizon (Theorem 8.3.1) and, the more finance-relevant case, a finite horizon with a terminal reward (Theorems 8.3.2-8.3.3).

Significance

Piecewise Deterministic Markov Processes have no substrate anywhere in Mathlib or on the platform, and this mission's content is a genuine method, not just a specialization of existing results: it shows how to reduce an entire continuous-time control problem to the discrete-time theory already available, at the cost of enlarging the state of the discrete embedded problem's own data (which becomes an integral against a whole flow, not a pointwise value) and enlarging the control space when compactness is needed for existence. This mission is careful to keep two distinctions the book itself insists on separate: relaxed versus nonrelaxed controls (Theorem 8.2.6 produces only the former; recovering the latter is Theorem 8.2.7's own, harder, conditional content), and the general Piecewise Deterministic model of §8.1-8.2 versus the simpler, discrete-state Markov Decision Chain of §8.3, which is a genuinely different structure (sums over a countable state space rather than integrals against a controlled flow), not an instance of the general model specialized after the fact.

Difficulty

The central formalization challenge is representing the continuous-time expected reward Vπ(x)V^\pi(x)Vπ(x) (Eq. (8.2)) faithfully, since the book itself only asserts the existence of a probability space carrying the jump-time/post-jump-state process with a specified conditional law, citing general marked-point-process theory rather than constructing it. This mission builds that probability-space data directly — via Mathlib's conditional expectation, conditioning on the current post-jump state — rather than treating VπV^\piVπ as a bare hypothesis-only quantity, so that Theorem 8.2.1's value-equality claim is a genuine, non-vacuous correspondence between two independently-defined objects (a continuous-time path functional and a discrete-time recursion) rather than true by definitional fiat. A second, compounding difficulty is that the same correspondence must be built twice — once for the general flow-driven process (§8.2) and again, independently, for the discrete-state jump process of the finite-horizon chain (§8.3) — since the two models share no state-space structure. A third difficulty specific to Theorem 8.2.8 is that its Hamilton-Jacobi-Bellman verification argument is genuinely differential (a process generator built from a gradient, a self-referential closed-loop control solving its own ODE), unlike every other result in this chapter, which works through the discrete-time embedding alone.

Formalization scope

The book's own topology on the control-function space AAA (the coarsest making certain integrals measurable) and the Young topology on the relaxed-control space RRR (which the book cites as making RRR separable, metric and compact, without reconstructing it) are not rebuilt from scratch; continuity/compactness hypotheses that need a topology on these spaces are stated directly against the pointwise/product topology on the underlying function types, a faithful but representationally simpler stand-in documented in MODERATION_NOTES.md. The embedded kernel Q′Q'Q′ (Eqs. (8.4), (8.7)) is bundled as data satisfying its own defining integral identity rather than literally constructed as a mixture of pushforward measures — a routine but heavy argument that would add no mathematical content beyond the formula itself. This mission's own goal (Theorem 8.2.6) explicitly produces a relaxed-control optimal policy, not a nonrelaxed one: stating it with a UUU-valued policy instead would silently substitute Theorem 8.2.7's strictly harder, conditionally-true conclusion for Theorem 8.2.6's own unconditional one, exactly the trivializing formalization this chapter's own structure warns against.

Selected references

  • N. Bäuerle and U. Rieder, Markov Decision Processes with Applications to Finance, Universitext, Springer, 2011. DOI: 10.1007/978-3-642-18324-9.
  • M. H. A. Davis, Markov Models and Optimization, Chapman & Hall, 1993 (the standard reference for Piecewise Deterministic Markov Processes, cited by the book for extensions of this chapter's basic model).
  • A. A. Yushkevich, "On reducing a jump controllable Markov model to a model with discrete time", Theory of Probability and its Applications, 1980 (the topology on the control-function space AAA cited by Definition 8.1.1's own construction).
  • H. J. Kushner and P. G. Dupuis, Numerical Methods for Stochastic Control Problems in Continuous Time, 2nd ed., Springer, 2001 (the Young topology and the Chattering Theorem, cited by Remark 8.2.3).
  • N. Bäuerle and U. Rieder, "Optimal control of piecewise deterministic Markov processes with finite time horizon", in Modern Trends of Controlled Stochastic Processes: Theory and Applications, 2010 (cited for the finite-horizon extension of this chapter's model, applied in chunk 09b's Sections 9.3-9.4).
16 thms2 active usersReviewed
Convex OptimizationDiscrete GeometryOptimization·Captain: Shuze Chen

Discrete Convex Analysis VI: Quasi M-Convex Functions and the Quasi-Proximity TheoremTextbook

Motivation

Convexity is normally defined additively — a function's value at a mixture is bounded by the mixture of its values — but many of the properties that make convexity useful in optimization (a local minimum is global, level sets are well-behaved) survive under a much weaker, purely ordinal notion: quasi-convexity, which compares function values rather than adding them. A nondecreasing rescaling of a convex function is generally not convex, but it is always quasi-convex — so a theory built only on ordinal comparisons automatically covers every such rescaling for free, at the cost of a more delicate proof architecture (since the algebraic cancellations available to additive convexity are no longer available).

Chapter 6's second half asks exactly how far this idea extends in the discrete setting: does the M-convexity exchange axiom have an ordinal, quasi-convex relaxation that still supports the same strong minimization theory — an optimality criterion, a minimizer-cut lemma, and, most significantly, a proximity theorem with the same explicit distance bound? This mission formalizes the chapter's answer: yes, and the relevant relaxed class, functions satisfying condition (SSQM≠_{\ne}=​), is large enough to include every strictly increasing rescaling of an M-convex function, a class the M-convex theory of chunk 06 alone says nothing about.

Setting

Let VVV be a finite ground set and f:ZV→R∪{+∞}f : \mathbb Z^V \to \mathbb R \cup \{+\infty\}f:ZV→R∪{+∞} with nonempty effective domain. Building on chunk 06's M-convex exchange axiom (M-EXC[Z]), this chapter introduces several ordinal relaxations. fff is weakly quasi M-convex, satisfying (QMw), if for every pair of distinct points x,y∈dom⁡fx, y \in \operatorname{dom} fx,y∈domf there exist uuu in the positive support and vvv in the negative support of x−yx - yx−y with f(x−χu+χv)≤f(x)f(x - \chi_u + \chi_v) \le f(x)f(x−χu​+χv​)≤f(x) or f(y+χu−χv)≤f(y)f(y + \chi_u - \chi_v) \le f(y)f(y+χu​−χv​)≤f(y) — an "or" where (M-EXC[Z]) demands an additive inequality. Two further conditions restrict attention to points of different function value and sharpen the conclusion to a three-way trichotomy (strictly better on one side, or exactly tied on both): (SSQM≠_{\ne}=​) quantifies universally over uuu (as in (M-EXC[Z])), while (SSQM≠,w_{\ne,w}=,w​) quantifies existentially over both uuu and vvv (as in (QMw)). The linear perturbation of fff by p:V→Rp : V \to \mathbb Rp:V→R is f[p](x)=f(x)−⟨p,x⟩f[p](x) = f(x) - \langle p, x \ranglef[p](x)=f(x)−⟨p,x⟩.

Formalization targets

Goal: Theorem 6.78 (the quasi M-proximity theorem)

Let fff satisfy (SSQM≠_{\ne}=​), n=∣V∣n = |V|n=∣V∣, α\alphaα a positive integer. If xα∈dom⁡fx_\alpha \in \operatorname{dom} fxα​∈domf satisfies f(xα)≤f(xα+α(χv−χu))f(x_\alpha) \le f(x_\alpha + \alpha(\chi_v - \chi_u))f(xα​)≤f(xα​+α(χv​−χu​)) for all u,v∈Vu, v \in Vu,v∈V, then arg⁡min⁡f≠∅\arg\min f \ne \emptysetargminf=∅ and there is x∗∈arg⁡min⁡fx^* \in \arg\min fx∗∈argminf with ∥xα−x∗∥∞≤(n−1)(α−1)\|x_\alpha - x^*\|_\infty \le (n-1)(\alpha - 1)∥xα​−x∗∥∞​≤(n−1)(α−1) — verbatim the same conclusion, and the same exact bound, as chunk 06's Theorem 6.37(1), now established for the strictly larger class satisfying (SSQM≠_{\ne}=​) rather than the M-convex exchange axiom itself.

Milestones: Theorems 6.68(2), 6.76, 6.77

Theorem 6.68(2): fff satisfies (M-EXC[Z]) if and only if every linear perturbation f[p]f[p]f[p] satisfies (QMw) — quantifying exactly how much weaker (QMw) is pointwise, and how the gap closes once quantified over every perturbation. Theorem 6.76 (the quasi M-optimality criterion): the direct analogue of chunk 06's Theorem 6.26 for the quasi-convexity classes — a purely pairwise local check still characterizes global (or, in the (QMw) case, strict unique) optimality. Theorem 6.77 (the quasi M-minimizer cut): chunk 06's Theorem 6.28 continues to hold verbatim when its M-convexity hypothesis is replaced by (SSQM≠_{\ne}=​) — the structural fact the proximity theorem's proof is built from survives the relaxation intact.

Significance

The result itself. The proximity theorem is the result algorithms actually use: a scaling algorithm for minimizing quasi-convex functions of this kind inherits exactly the same correctness guarantee, with exactly the same distance bound, as the M-convex case — this is a genuine broadening of chapter 10's algorithmic reach, not a restatement dressed in weaker hypotheses. Every strictly increasing scalar transformation of an M-convex objective (a common modeling device — re-expressing a cost in utility units, or applying a monotone risk measure) now falls under a proximity theorem, whereas prior to this chapter's relaxation such a transformation would generally destroy M-convexity itself and leave optimization theory silent on the transformed problem.

Formalizing it. No matching item exists on the platform for quasi M-convexity in any of its forms. Formalizing Theorem 6.78 requires first pinning down (SSQM≠_{\ne}=​) exactly (there are six closely related axiom variants in this section of the book, only three of which — (QMw), (SSQM≠_{\ne}=​), (SSQM≠,w_{\ne,w}=,w​) — are needed for this mission's chosen results), and this mission also captures, via Theorem 6.68(2), the precise sense in which these relaxed conditions are strictly weaker than plain M-convexity while remaining tightly connected to it.

Difficulty

The natural first instinct, given how close the quasi-convexity axioms look to (M-EXC[Z]), is to try to prove Theorem 6.78 by directly imitating chunk 06's proof of Theorem 6.37 line by line. This mostly works — the proof structure (fix a target coordinate, build a chain of strictly decreasing values via repeated exchange steps, bound the chain's length using the scaled hypothesis) survives verbatim — but every step that chunk 06's proof took by adding two instances of the exchange inequality together must be replaced by an ordinal argument, since (SSQM≠_{\ne}=​) only ever asserts a disjunction of value comparisons, never an additive inequality relating four function values simultaneously the way (M-EXC[Z])'s f(x)+f(y)≥f(x−χu+χv)+f(y+χu−χv)f(x)+f(y) \ge f(x-\chi_u+\chi_v)+f(y+\chi_u-\chi_v)f(x)+f(y)≥f(x−χu​+χv​)+f(y+χu​−χv​) does. The book's proof handles this by working with strict inequalities and the trichotomy structure of (SSQM≠_{\ne}=​) directly rather than algebraic cancellation — the same overall architecture, but every arithmetic step rebuilt as a case analysis on which disjunct of (SSQM≠_{\ne}=​) fires.

Formalization scope

This mission builds directly on chunk 06's published items (CharVec, SuppPos, SuppNeg, DomZ, MExchangeAxiom, ArgMin), per the platform's textbook convention that a later chapter of the same book imports an earlier one's definitions rather than redrafting them; its own namespace DiscreteConvex.MConvexFunctions.Quasi nests under chunk 06's DiscreteConvex.MConvexFunctions accordingly. Δf(z;v,u) (Eq. (6.2)) is never reified as a separate object; every occurrence is unfolded directly into an f-value comparison, avoiding WithTop ℝ subtraction throughout, consistent with chunk 06's own convention.

A trivializing formalization of the goal would silently strengthen (SSQM≠_{\ne}=​) back to plain M-convexity (making this mission redundant with chunk 06's Theorem 6.37) or loosen the exact bound (n−1)(α−1)(n-1)(\alpha-1)(n−1)(α−1) to an unspecified function of n,αn, \alphan,α; neither is done. Six axiom variants appear in this section of the book ((QM), (SSQM), (QMw), (SSQMw_ww​), (SSQM≠_{\ne}=​), (SSQM≠,w_{\ne,w}=,w​)); only the three actually needed by this mission's four items are drafted, and Theorem 6.68's first part (an implication chain among the other three) is left out — see MODERATION_NOTES.md. Contributions building the polyhedral M-convex-function bridge (§6.11–6.12, Theorems 6.59–6.64), the level-set characterizations (Theorems 6.72, 6.74), or the scaled quasi M-minimizer cut (Theorem 6.79, the direct generalization of Theorem 6.77 drafted here) are welcome.

Selected references

  • K. Murota, Discrete Convex Analysis, SIAM, 2003. DOI: 10.1137/1.9780898718508.
  • M. Avriel, W. E. Diewert, S. Schaible, I. Zang, Generalized Concavity, Plenum Press, 1988.
8 thms2 active usersReviewed
Dynamic ProgrammingOptimizationProbability·Captain: Shuze Chen

Markov Decision Processes XV: Optimal Play in Red-and-Black and the Gittins IndexTextbook

Motivation

Chapter 7's abstract machinery — contracting Markov Decision Models, the Structure Theorem, value iteration with an explicit convergence rate — earns its keep by solving concrete problems. Section 7.6 works through four kinds of application: a return to the classical cash-balance inventory problem, now over an infinite horizon; the "red-and-black" gambling problem, where a player tries to reach a target fortune before going bankrupt; and, most substantially, the infinite-horizon two-armed bandit, where the general theory reveals something genuinely surprising — the qualitatively optimal policy can be computed one arm at a time.

Setting

Every application here specializes the general infinite-horizon, contracting-model machinery of chunks 07a/07b to a concrete transition structure. The cash-balance model orders inventory up to a level aaa at linear cost, incurs a holding/shortage cost, then absorbs a random demand. The red-and-black model bets a fraction of a bounded fortune on a biased coin, absorbing at bankruptcy or at the target. The bandit model reconsiders the Beta-Bernoulli two-armed bandit of chunk 05b, now over an infinite horizon with a genuine discount β<1\beta<1β<1: the key new tool is the K-stopping problem, a fictitious single-arm decision problem where, at every stage, the decision maker may either pull the arm or retire with a fixed payment KKK. The Gittins index I(m,n)I(m,n)I(m,n) is the smallest such payment at which retiring immediately is already as good as continuing.

Formalization targets

The goal, Theorem 7.6.10, is the Gittins index theorem for this book's two-armed bandit: always pulling the arm with the higher index is optimal for the full infinite-horizon problem. The milestones build the machinery it needs — the index's definition (Definition 7.6.5) and its equivalent representation as a supremum over stopping times (Theorem 7.6.6), the K-stopping value function's monotonicity/convexity/differentiability properties (Proposition 7.6.7), the index's optimal-stopping-set and indifference characterizations (Corollary 7.6.8), the two-arm joint stopping value's parallel structure (Proposition 7.6.9), and a fixed-point recasting useful for computation (Proposition 7.6.11) — plus, independently, the cash-balance and casino-game applications (Theorems 7.6.1-7.6.4), which use the general theory but not the bandit-specific machinery.

Significance

The Gittins index theorem's real content, emphasized by the book's own remark, is not merely that an optimal policy exists but how little computation it needs: instead of solving one optimization problem over the bandit's full four-dimensional joint state space N02×N02\mathbb N_0^2 \times \mathbb N_0^2N02​×N02​, the decision maker solves two independent two-dimensional single-arm problems and compares two numbers. This mission's formalization of the goal is built specifically to keep that separation visible — each arm's index is computed from a single, shared KStoppingValue structure applied to that arm's own state alone, never from a function that happens to take the whole joint state as an argument. The proof route here (via the K-stopping problem's explicit fixed-point characterization, Definition 7.6.5 and Proposition 7.6.11) is a genuinely different construction from the platform's existing Gittins-index theorems (BanditAlgorithm.gittins_index_theorem and related), which are built via Whittle's retirement/charge-accounting argument — checked directly and found to define the index differently enough that this mission drafts its own theorems rather than treat that construction as prior art.

Difficulty

The K-stopping value function J(m,n;K)J(m,n;K)J(m,n;K) and the two-arm joint value J~(x;K)\tilde J(x;K)J~(x;K) are both genuine fixed points of an infinite-horizon Bellman equation with no finite backward recursion to fall back on (the "stopping" option, rather than a terminal condition, is what makes the horizon infinite); this mission bundles them as data satisfying their own defining fixed-point equations, the same convention this series uses throughout for such objects. A second difficulty is Theorem 7.6.6's supremum over stopping times: without a canonical path measure for the underlying Markov chain (not built anywhere in this series), the two expectations the theorem compares are represented as data satisfying the positivity a genuine expectation must have, over an explicit, elementary notion of stopping time (a function of the whole observed path, adapted in the sense that whether it has fired by time nnn depends only on the path up to nnn) — a faithful, if representational, rendering of the theorem's genuinely path-dependent content.

Formalization scope

The cash-balance model (Theorem 7.6.1) explicitly cites chunk 02d's finite-horizon critical-level sequences as a hypothesis rather than re-deriving them, since this mission's own content is the infinite-horizon extension, not a second proof of the finite-horizon theory those sequences come from. The casino-game theorems (7.6.2-7.6.4) state optimality for the specific, named timid and bold strategies, not for an unnamed "some optimal policy" — the theorems' entire content is that these particular policies, not merely some optimal one, are best in their regime. The bandit model's posterior mean and Bayes-update operator are kept identical in substance to chunk 05b's finite-horizon Beta-Bernoulli model (restated, since chunks cannot import each other's Lean), so a reader can see this section is solving the same underlying statistical model, now over an infinite horizon.

Selected references

  • N. Bäuerle and U. Rieder, Markov Decision Processes with Applications to Finance, Universitext, Springer, 2011. DOI: 10.1007/978-3-642-18324-9.
  • J. C. Gittins, "Bandit processes and dynamic allocation indices," Journal of the Royal Statistical Society, Series B, 1979 (the original index construction this section's K-stopping-problem approach reformulates).
  • P. Whittle, "Multi-armed bandits and the Gittins index," Journal of the Royal Statistical Society, Series B, 1980 (the retirement-option construction the platform's existing Gittins theorems use, a different proof route from this chunk's own).
  • L. E. Dubins and L. J. Savage, How to Gamble If You Must: Inequalities for Stochastic Processes, McGraw-Hill, 1965 (the classical red-and-black problem, Theorems 7.6.2-7.6.4).
14 thms2 active usersReviewed
🏆Completed
Convex OptimizationDiscrete GeometryOptimization·Captain: Shuze Chen

Discrete Convex Analysis V: The M-Optimality Criterion and the Proximity TheoremTextbook

Motivation

Scaling algorithms are one of the standard techniques for solving discrete optimization problems efficiently: instead of searching a huge integer domain directly, an algorithm first solves a coarsened version of the problem — checking optimality only against neighbors reached by a large step size α\alphaα — and then refines the resulting approximate solution down to the true optimum. This strategy is only as good as the guarantee that a coarse-scale local optimum is provably close to a true, fine-scale global optimum; without such a guarantee, refinement could require an unbounded number of steps. Results providing this guarantee are called proximity theorems, and they are a standard tool across combinatorial optimization, from network flow scaling algorithms to submodular function minimization.

M-convex functions, the subject of this chapter, are exactly the class of discrete convex functions for which the classical local-optimality test of chapter 3 (checking a full neighborhood of up to 3n−13^n-13n−1 sign patterns) sharpens to a much smaller, purely pairwise test: checking f(x)≤f(x−χu+χv)f(x) \le f(x - \chi_u + \chi_v)f(x)≤f(x−χu​+χv​) for every pair of coordinates u,vu, vu,v. This mission formalizes the chapter's central definitional equivalence (Theorem 6.2), this pairwise optimality criterion (Theorem 6.26), a structural minimizer-cut lemma (Theorem 6.28), and the chapter's capstone, the M-proximity theorem (Theorem 6.37) — the result that makes M-convex scaling algorithms provably correct, with an explicit, dimension-and-scale-only distance bound between a coarse-scale local optimum and a true global minimizer.

Setting

Let VVV be a finite ground set. A function f:ZV→R∪{+∞}f : \mathbb Z^V \to \mathbb R \cup \{+\infty\}f:ZV→R∪{+∞} with nonempty effective domain dom⁡f\operatorname{dom} fdomf is an M-convex function if it satisfies the exchange axiom (M-EXC[Z]): for x,y∈dom⁡fx, y \in \operatorname{dom} fx,y∈domf and uuu in the positive support of x−yx - yx−y, there is vvv in the negative support of x−yx-yx−y with

f(x)+f(y)≥f(x−χu+χv)+f(y+χu−χv).f(x) + f(y) \ge f(x - \chi_u + \chi_v) + f(y + \chi_u - \chi_v).f(x)+f(y)≥f(x−χu​+χv​)+f(y+χu​−χv​).

An M♮^\natural♮-convex function is one whose lift f~\tilde ff~​ to the extended ground set V~={0}∪V\tilde V = \{0\} \cup VV~={0}∪V — defined by f~(x0,x)=f(x)\tilde f(x_0, x) = f(x)f~​(x0​,x)=f(x) when x0=−x(V)x_0 = -x(V)x0​=−x(V), and +∞+\infty+∞ otherwise — is M-convex; equivalently (Theorem 6.2, below) fff satisfies the axiom (M♮^\natural♮-EXC[Z]), a variant of (M-EXC[Z]) that additionally allows a single-coordinate move (uuu alone, with no compensating vvv). Every M-convex function is M♮^\natural♮-convex, but not conversely. For α\alphaα a positive integer, a point satisfies the scaled local optimality condition at scale α\alphaα if f(xα)≤f(xα+α(χv−χu))f(x_\alpha) \le f(x_\alpha + \alpha(\chi_v - \chi_u))f(xα​)≤f(xα​+α(χv​−χu​)) for all relevant u,vu, vu,v — a check against neighbors α\alphaα steps away rather than adjacent ones.

Formalization targets

Goal: Theorem 6.37 (the M-proximity theorem)

Assume α\alphaα is a positive integer and n=∣V∣n = |V|n=∣V∣.

(1) f M-convex, f(xα)≤f(xα+α(χv−χu)) ∀u,v  ⟹  ∃x∗∈arg⁡min⁡f, ∥xα−x∗∥∞≤(n−1)(α−1),\text{(1) } f \text{ M-convex, } f(x_\alpha) \le f(x_\alpha + \alpha(\chi_v-\chi_u))\ \forall u,v \implies \exists x^* \in \arg\min f,\ \|x_\alpha - x^*\|_\infty \le (n-1)(\alpha-1),(1) f M-convex, f(xα​)≤f(xα​+α(χv​−χu​)) ∀u,v⟹∃x∗∈argminf, ∥xα​−x∗∥∞​≤(n−1)(α−1), (2) f M♮-convex, same hypothesis over u,v∈V∪{0}  ⟹  ∃x∗∈arg⁡min⁡f, ∥xα−x∗∥∞≤n(α−1).\text{(2) } f \text{ M}^\natural\text{-convex, same hypothesis over } u,v \in V \cup \{0\} \implies \exists x^* \in \arg\min f,\ \|x_\alpha - x^*\|_\infty \le n(\alpha-1).(2) f M♮-convex, same hypothesis over u,v∈V∪{0}⟹∃x∗∈argminf, ∥xα​−x∗∥∞​≤n(α−1).

Both bounds are exact and specific to their hypothesis class; replacing either with an unspecified function of nnn and α\alphaα would discard exactly the content chapter 10's algorithms rely on.

Milestones: Theorems 6.2, 6.26, 6.28

Theorem 6.2: M♮^\natural♮-convexity (defined via the lift) is equivalent to the direct exchange axiom (M♮^\natural♮-EXC[Z]) — the chapter's central definitional theorem, needed to work with M♮^\natural♮-convex functions without repeatedly invoking the lift construction. Theorem 6.26 (the M-optimality criterion): global optimality of fff at xxx is equivalent to a purely pairwise local check, f(x)≤f(x−χu+χv)f(x) \le f(x-\chi_u+\chi_v)f(x)≤f(x−χu​+χv​) for all u,vu,vu,v (plus, in the M♮^\natural♮ case, f(x)≤f(x±χv)f(x) \le f(x\pm\chi_v)f(x)≤f(x±χv​)). Theorem 6.28 (the M-minimizer cut): from any point and any coordinate pair minimizing a one-step exchange, one can certify a coordinate-wise bound that some global minimizer must satisfy — the structural fact underlying both the domain-reduction algorithm and, via the same proof technique, the proximity theorem itself.

Significance

The result itself. Theorem 6.26 already sharpens chapter 3's local-to-global criterion (checking a full 3n−13^n-13n−1-point neighborhood) to an O(n2)O(n^2)O(n2)-size pairwise check — the minimum spanning tree optimality criterion is a direct special case. The proximity theorem builds on this to control what happens when the local check is only performed at a coarse scale α\alphaα: it guarantees that scaling-based algorithms, which alternate between coarse-scale local search and scale reduction, terminate with a guaranteed-close approximation at every stage, with an explicit linear-in-nnn, linear-in-α\alphaα error bound rather than a qualitative "eventually converges" guarantee.

Formalizing it. No matching item exists on the platform for M-convex functions, the exchange axiom, or a discrete proximity theorem of this kind. This mission gives the first formal statement of the M-optimality criterion and the M-proximity theorem, together with the exchange-axiom / lift-based-definition equivalence (Theorem 6.2) that the rest of the M-convex function theory (chunks 07, and indirectly 10–14) is built on.

Difficulty

The natural first attempt at Theorem 6.37 is to try a direct coordinatewise argument: since the scaled hypothesis holds for every pair u,vu, vu,v, one might hope to bound ∣xα(v)−x∗(v)∣|x_\alpha(v) - x^*(v)|∣xα​(v)−x∗(v)∣ coordinate by coordinate independently. This does not work, because a single application of the exchange axiom only ever improves fff by trading one coordinate down and one other coordinate up simultaneously — there is no way to move a single coordinate toward a minimizer in isolation without accounting for where the compensating mass goes. The actual proof instead fixes a target coordinate vvv, constructs a chain of strictly decreasing function values y0=xα,y1,…,yky_0 = x_\alpha, y_1, \ldots, y_ky0​=xα​,y1​,…,yk​ by repeatedly applying (M-EXC[Z]) against a fixed near-optimal point x∗x^*x∗ (exactly the technique of Theorem 6.28's proof), and then bounds how far each other coordinate can move along this chain using the scaled hypothesis itself, before summing those bounds via the M-convex domain's hyperplane constraint x(V)=x(V) = x(V)= constant to recover the bound on vvv. The chain construction, not a per-coordinate estimate, is what makes the linear-in-nnn bound provable at all.

Formalization scope

The ground set VVV is a Fintype with DecidableEq; f:ZV→R∪{+∞}f : \mathbb Z^V \to \mathbb R \cup \{+\infty\}f:ZV→R∪{+∞} is (V → ℤ) → WithTop ℝ. M♮^\natural♮-convexity is represented via an explicit lift to Option V (none standing for the extended ground set's new element 000), matching the book's own primary definition; the direct exchange-axiom form is a separate predicate related to it by Theorem 6.2, not conflated with it. `‖x_\alpha - x^*|_\infty \le c$ is stated pointwise.

A trivializing formalization of the goal would replace either exact bound, (n−1)(α−1)(n-1)(\alpha-1)(n−1)(α−1) or n(α−1)n(\alpha-1)n(α−1), with an unspecified asymptotic bound, or merge the two hypothesis classes into a single weaker statement; neither is done here. Propositions establishing dom f as an M-convex set, the M/M♮^\natural♮ relationship (Theorem 6.3), and several structural closure properties are cut from this mission's scope (not needed by the chosen items' statements — see MODERATION_NOTES.md) but are natural targets for a follow-on mission or for chunk 07, which builds directly on this chunk's exchange-axiom vocabulary. Contributions building the arg min f M-convexity corollary (Proposition 6.29) or the scaled minimizer cut (Theorem 6.39, the direct generalization of Theorem 6.28 drafted here) are welcome.

Selected references

  • K. Murota, Discrete Convex Analysis, SIAM, 2003. DOI: 10.1137/1.9780898718508.
  • D. S. Hochbaum, "Lower and upper bounds for the allocation problem and other nonlinear optimization problems," Mathematics of Operations Research, 19(2), 1994, pp. 390–409.
14 thms2 active usersReviewed
🏆Completed
Dynamic ProgrammingOptimizationProbability·Captain: Shuze Chen

Markov Decision Processes XIII: Contracting Infinite-Horizon Markov Decision ModelsTextbook

Motivation

Every mission in this series so far has treated a finite-horizon decision problem: an investor or planner with a fixed, known number of periods left. Many of the most important applications — perpetual investment, an infinitely-repeated inventory or maintenance problem, a firm that never stops operating — have no natural end date at all. Chapter 7 of Bäuerle and Rieder's Markov Decision Processes with Applications to Finance (Springer, 2011) builds the theory needed to make sense of "the value of a decision problem that never ends," and does so on a general Borel state space rather than a finite one. This mission covers the chapter's first three sections: the general infinite-horizon setup, the semicontinuous existence theory that makes it usable, and the sharper contraction-based theory that is the chapter's, and arguably the whole book's, theoretical center.

Setting

An infinite-horizon Markov Decision Model reuses the same data (E,A,D,Q,r,β)(E,A,D,Q,r,\beta)(E,A,D,Q,r,β) as the finite-horizon models of earlier chapters, but drops the terminal reward and applies a single (possibly randomized) decision rule at every one of infinitely many stages. Its performance criterion, J∞(x):=sup⁡πExπ[∑k=0∞βkr(Xk,fk(Xk))]J_\infty(x) := \sup_\pi \mathbb E^\pi_x\big[\sum_{k=0}^\infty \beta^kr(X_k,f_k(X_k)) \big]J∞​(x):=supπ​Exπ​[∑k=0∞​βkr(Xk​,fk​(Xk​))], is only meaningful once an integrability condition (Assumption (A)) and a convergence condition (Assumption (C)) rule out the sum diverging or the finite-horizon approximations failing to settle down. Both conditions follow automatically once the model has an upper bounding function bbb — a function controlling both the size of the reward and how fast the transition kernel can grow bbb itself — with βαb<1\beta\alpha_b < 1βαb​<1, the manageable special case that covers both the classical bounded-reward discounted case and the case of a non-positive reward.

Formalization targets

The goal, Theorem 7.3.5 (Structure Theorem), is the chapter's capstone: under a genuine bounding function (a two-sided reward bound making the space IBbIB_bIBb​ of finite-weighted-norm functions a Banach space) with βαb<1\beta\alpha_b < 1βαb​<1, and one abstract structural hypothesis — a closed class IM⊂IBbIM \subset IB_bIM⊂IBb​ containing 000, mapped into itself by the Bellman operator TTT, on which a maximizing action always exists — Banach's fixed point theorem delivers existence, uniqueness, an explicit geometric convergence rate for value iteration, and existence of an optimal stationary policy, all at once. The milestones build up to it in three stages: the general infinite-horizon machinery (Lemmas 7.1.4-7.1.5, Theorems 7.1.6-7.1.8 — reward iteration, a verification theorem, and a structure theorem under an abstract structure assumption that is not yet tied to any checkable property of the model); the semicontinuous existence theory that gives primitive, checkable conditions implying that abstract assumption (Theorem 7.2.1 and its two corollaries, including a genuine policy iteration conclusion); and the contracting theory proper (Lemma 7.3.3's contraction estimate, Theorem 7.3.4's sharpened verification theorem, and Theorem 7.3.6's continuous specialization of the goal).

Significance

The goal is the direct, general-Borel-space generalization of what finite-state dynamic programming theorems already on the platform (BertsekasDP.discounted_main_theorem, BertsekasDP.ssp_main_theorem) establish only for a finite state and action space, where Banach's theorem is applied directly on Rn\mathbb R^nRn: this mission's content is that the same conclusions — including the same explicit geometric convergence rate for value iteration — hold on an arbitrary Borel state space, the moment one abstract, structural condition is checked. That condition is not vacuous or automatic: Example 7.2.4 (cited but not itself formalized, being an unnumbered worked counterexample rather than a numbered result) shows that without compactness of the feasible-action correspondence, the naive Structure Assumption of Chapter 2 is not enough and value iteration can converge to the wrong limit (J≠J∞J \ne J_\inftyJ=J∞​). Theorems 7.1.8's Structure Assumption (SA) is built precisely to rule this out, and Theorem 7.2.1's semicontinuity/ compactness conditions are the practical, checkable sufficient conditions for it.

Difficulty

The central formalization challenge is representing J∞πJ_\infty^\piJ∞π​, the genuine infinite-horizon expected discounted reward, without constructing a canonical infinite-horizon path measure from the model's transition kernel — a substantial undertaking the book itself sidesteps by proving (via an appendix result, Theorem B.1.1, not itself reproved here) that J∞πJ_\infty^\piJ∞π​ equals the limit of the finite-horizon truncations JnπJ_n^\piJnπ​. This mission takes that limit characterization as its own definition, via Filter.limsup for the same total, junk-safe reasons this book's series has used throughout (no canonical path measure anywhere in chunks 02a, 05a, 05b, 06). A second, genuinely new difficulty is Ls, the "upper limit of a sequence of sets" that drives every policy-iteration conclusion: it is a statement about accumulation points of a sequence of points, one drawn from each set in the sequence, not the more familiar set-theoretic limsup of a sequence of sets — getting this distinction right is the entire content of what "policy iteration" asserts.

Formalization scope

Every operator, bounding-function class, and value function of §7.1-7.3 is restated (not imported) in this chunk's own namespace from the finite-horizon originals of chunks 02a/02b, adapted to drop the time index and bake the discount into the one-stage operator directly, per this chapter's own presentation. The contracting theory's genuinely real-valued Banach-space objects (Tf′T_f'Tf′​, T′T'T′, IBbIB_bIBb​, the weighted norm ∥⋅∥b\|\cdot\|_b∥⋅∥b​) are kept separate from the general theory's EReal-valued objects (TfT_fTf​, TTT, IM(E)IM(E)IM(E)), matching the book's own distinction between a value function that is a priori only known to avoid +∞+\infty+∞ and one known to be genuinely finite everywhere. Part (d) of the goal — the explicit geometric convergence rate — is stated in full, not weakened to bare qualitative convergence, since a formalization that dropped it would lose exactly the fact (used again by this book's own Theorem 7.5.12, a different chunk) that makes value iteration a genuine numerical method with a computable error bound rather than merely an existence argument.

Selected references

  • N. Bäuerle and U. Rieder, Markov Decision Processes with Applications to Finance, Universitext, Springer, 2011. DOI: 10.1007/978-3-642-18324-9.
  • D. P. Bertsekas, Dynamic Programming and Optimal Control, Vol. II, 4th ed., Athena Scientific, 2012 (the finite-state discounted/SSP theorems this goal generalizes).
  • D. P. Bertsekas and S. E. Shreve, Stochastic Optimal Control: The Discrete-Time Case, Academic Press, 1978 (analytic measurability of J∞J_\inftyJ∞​, JJJ; cited by the book for this chapter's foundational measure-theoretic facts).
  • K. Hinderer, Foundations of Non-stationary Dynamic Programming with Discrete Time Parameter, Lecture Notes in Operations Research and Mathematical Systems 33, Springer, 1970 (Theorem 18.4, cited for the fact that history-dependent policies do not improve on Markov ones).
16 thms2 active usersReviewed
🏆Completed
Convex OptimizationMachine LearningOptimization·Captain: mikedeng1

A Stochastic Quasi-Newton Method for Large-Scale Optimization: The Expected Suboptimality Bound of the SQN MethodResearch Paper

Motivation

Training a statistical model by empirical risk minimization means minimizing an average of NNN losses over a parameter vector w∈Rnw\in\mathbb R^nw∈Rn, where both NNN and nnn can be in the millions. Stochastic gradient descent (SGD) is the standard method: each step uses the gradient of a small random batch of losses. It is cheap per step but sensitive to the scaling of the problem. Quasi-Newton methods such as L-BFGS correct the scaling in deterministic optimization, but naive stochastic versions are unstable, because differences of noisy gradients are poor curvature estimates.

Byrd, Hansen, Nocedal and Singer (SIAM J. Optim. 26(2), 2016) proposed the stochastic quasi-Newton (SQN) method. It decouples the two estimates: gradients come from small batches at every step, while curvature pairs are formed only every LLL steps, from averaged iterates and subsampled Hessian–vector products. The paper's analysis (Section 3) shows that the method keeps the O(1/k)O(1/k)O(1/k) expected suboptimality rate of SGD on strongly convex problems. The same eigenvalue bound was obtained independently by Mokhtari and Ribeiro (JMLR 16, 2015). Bottou, Curtis and Nocedal later gave a general treatment of such preconditioned stochastic methods (SIAM Review 60(2), 2018).

Setting

Let f1,…,fN:Rn→Rf_1,\dots,f_N:\mathbb R^n\to\mathbb Rf1​,…,fN​:Rn→R be twice continuously differentiable losses and

F(w)=1N∑i=1Nfi(w).F(w)=\frac1N\sum_{i=1}^N f_i(w).F(w)=N1​i=1∑N​fi​(w).

For a sample S⊆{1,…,N}\mathcal S\subseteq\{1,\dots,N\}S⊆{1,…,N} of size bbb, the minibatch gradient is ∇FS(w)=1b∑i∈S∇fi(w)\nabla F_{\mathcal S}(w)=\frac1b\sum_{i\in\mathcal S}\nabla f_i(w)∇FS​(w)=b1​∑i∈S​∇fi​(w). For a sample SH\mathcal S_HSH​ of size bHb_HbH​, the subsampled Hessian is ∇2FSH(w)=1bH∑i∈SH∇2fi(w)\nabla^2F_{\mathcal S_H}(w)=\frac1{b_H}\sum_{i\in\mathcal S_H}\nabla^2 f_i(w)∇2FSH​​(w)=bH​1​∑i∈SH​​∇2fi​(w).

Assumption 1 requires constants 0<λ,Λ0<\lambda,\Lambda0<λ,Λ with λI≺∇2FSH(w)≺ΛI\lambda I\prec\nabla^2F_{\mathcal S_H}(w)\prec\Lambda IλI≺∇2FSH​​(w)≺ΛI for every www and every Hessian sample. It also requires a bound γ2\gamma^2γ2 on the second moment of the stochastic gradient. w∗w^*w∗ denotes the minimizer of FFF.

Algorithm 2 turns correction pairs (sj,yj)(s_j,y_j)(sj​,yj​) into a matrix HtH_tHt​. With m~=min⁡{t,M}\tilde m=\min\{t,M\}m~=min{t,M}, it starts from stTytytTytI\frac{s_t^Ty_t}{y_t^Ty_t}IytT​yt​stT​yt​​I and applies the BFGS update

H←(I−ρjsjyjT)H(I−ρjyjsjT)+ρjsjsjT,ρj=1yjTsj,H\leftarrow(I-\rho_js_jy_j^T)H(I-\rho_jy_js_j^T)+\rho_js_js_j^T,\qquad\rho_j=\frac1{y_j^Ts_j},H←(I−ρj​sj​yjT​)H(I−ρj​yj​sjT​)+ρj​sj​sjT​,ρj​=yjT​sj​1​,

for j=t−m~+1,…,tj=t-\tilde m+1,\dots,tj=t−m~+1,…,t.

Algorithm 1 (SQN) runs for k=1,2,…k=1,2,\dotsk=1,2,…. It draws a gradient sample Sk\mathcal S_kSk​ and steps

wk+1=wk−αkH∇FSk(wk),w^{k+1}=w^k-\alpha^kH\nabla F_{\mathcal S_k}(w^k),wk+1=wk−αkH∇FSk​​(wk),

where H=IH=IH=I for k≤2Lk\le2Lk≤2L and H=HtH=H_tH=Ht​, t=⌊(k−1)/L⌋−1t=\lfloor(k-1)/L\rfloor-1t=⌊(k−1)/L⌋−1, afterwards. Every LLL iterations it forms the block average wˉt\bar w_twˉt​ of the last LLL iterates and a new pair

st=wˉt−wˉt−1,yt=∇2FSH,t(wˉt) st.s_t=\bar w_t-\bar w_{t-1},\qquad y_t=\nabla^2F_{\mathcal S_{H,t}}(\bar w_t)\,s_t .st​=wˉt​−wˉt−1​,yt​=∇2FSH,t​​(wˉt​)st​.

Samples are drawn independently and uniformly among the subsets of their size. The step length is αk=β/k\alpha^k=\beta/kαk=β/k.

Formalization targets

Goal: Corollary 3.3, with a corrected constant

There are 0<μ1≤μ20<\mu_1\le\mu_20<μ1​≤μ2​, depending only on the problem data, such that every matrix Algorithm 1 applies satisfies μ1I≺H≺μ2I\mu_1I\prec H\prec\mu_2Iμ1​I≺H≺μ2​I. Moreover, for every β>1/(2μ1λ)\beta>1/(2\mu_1\lambda)β>1/(2μ1​λ),

E[F(wk)−F(w∗)]≤Qc(β)k(k≥1),E[F(w^k)-F(w^*)]\le\frac{Q_c(\beta)}k\qquad(k\ge1),E[F(wk)−F(w∗)]≤kQc​(β)​(k≥1), Qc(β)=max⁡{Λμ22β2γ22(2μ1λβ−1), Λμ22β2γ2, F(w1)−F(w∗)}.Q_c(\beta)=\max\Big\{\frac{\Lambda\mu_2^2\beta^2\gamma^2}{2(2\mu_1\lambda\beta-1)},\ \Lambda\mu_2^2\beta^2\gamma^2,\ F(w^1)-F(w^*)\Big\}.Qc​(β)=max{2(2μ1​λβ−1)Λμ22​β2γ2​, Λμ22​β2γ2, F(w1)−F(w∗)}.

The constants μ1,μ2\mu_1,\mu_2μ1​,μ2​ are not fixed numerically. The goal asserts a rate of order 1/k1/k1/k with an explicit constant in terms of them.

Milestones

  • (3.8)–(3.10): the curvature bounds λ∥s∥2≤yTs≤Λ∥s∥2\lambda\|s\|^2\le y^Ts\le\Lambda\|s\|^2λ∥s∥2≤yTs≤Λ∥s∥2 and λ≤∥y∥2/yTs≤Λ\lambda\le\|y\|^2/y^Ts\le\Lambdaλ≤∥y∥2/yTs≤Λ for Hessian-product pairs.
  • (3.11) and (3.12): the trace bound and Powell's determinant formula for the direct L-BFGS matrices.
  • Lemma 3.1: μ1I≺Ht≺μ2I\mu_1I\prec H_t\prec\mu_2Iμ1​I≺Ht​≺μ2​I uniformly along every run.
  • (3.18): expected descent for the general Newton-like iteration wk+1=wk−αkHk∇f(wk,ξk)w^{k+1}=w^k-\alpha^kH_k\nabla f(w^k,\xi^k)wk+1=wk−αkHk​∇f(wk,ξk).
  • (3.19): 2λ[F(w)−F(w∗)]≤∥∇F(w)∥22\lambda[F(w)-F(w^*)]\le\|\nabla F(w)\|^22λ[F(w)−F(w∗)]≤∥∇F(w)∥2.
  • (3.22): the recursion ϕk+1≤(1−2αkμ1λ)ϕk+Λ2(αkμ2)2γ2\phi_{k+1}\le(1-2\alpha^k\mu_1\lambda)\phi_k+\frac\Lambda2(\alpha^k\mu_2)^2\gamma^2ϕk+1​≤(1−2αkμ1​λ)ϕk​+2Λ​(αkμ2​)2γ2.
  • Theorem 3.2: the Qc(β)/kQ_c(\beta)/kQc​(β)/k rate for the Newton-like iteration.

Significance

The corollary says that curvature information costs nothing in rate. With uniformly bounded preconditioners and the β/k\beta/kβ/k schedule, the SQN method converges in expectation at the same order as SGD, which is the order known to be optimal for this class of problems. Lemma 3.1 is the reusable part: it holds for any L-BFGS matrix built from pairs whose curvature is controlled by a bounded Hessian. Theorem 3.2 applies to every stochastic method whose preconditioner is fixed before the sample is drawn and has uniformly bounded spectrum.

The formalization adds three things. First, it states the result correctly. As printed, Theorem 3.2 is false and Assumption 1(3) cannot be satisfied (see Formalization scope), and the mission states and labels the repaired versions. Second, it gives a machine-checked statement of the SQN algorithm itself, which is not currently formalized anywhere. Third, it provides L-BFGS and preconditioned-SGD infrastructure that later missions on stochastic second-order methods can reuse. To our knowledge none of these results has a machine-checked proof.

Difficulty

The natural first idea is to feed the iterates of Algorithm 1 to a standard SGD rate theorem. This fails for two reasons. The preconditioner HtH_tHt​ depends on the past iterates and on independent Hessian samples, so the analysis needs a filtration in which HtH_tHt​ is known before the gradient sample is drawn. Also, the rate proof itself is an induction that breaks in the first iterations, which is exactly where the printed argument goes wrong.

On the linear-algebra side, the difficulty is a lower bound on the smallest eigenvalue of HtH_tHt​ that is uniform over all runs and all ttt. The curvature bounds on each individual pair do not give it directly, because the BFGS updates compound across the memory window. The expectation side needs conditional expectations of vector-valued functions and a descent inequality under a Hessian bound, neither of which Mathlib packages for this setting.

Formalization scope

Vectors live in EuclideanSpace ℝ (Fin n), matrices are Matrix (Fin n) (Fin n) ℝ acting through Matrix.toEuclideanLin, and A≺BA\prec BA≺B is (B - A).PosDef. Hessians are fderiv ℝ (gradient f) w. Indices k,tk,tk,t start at 111 as in the paper. In the corollary, EEE is an exact finite average over sample histories, and "almost surely" means "on every history". Theorem 3.2 uses a general probability space with a filtration. HkH_kHk​ and wkw^kwk are Fk\mathcal F_kFk​-measurable, the sample ξk\xi^kξk is Fk+1\mathcal F_{k+1}Fk+1​-measurable, and unbiasedness is a conditional expectation. Integrability of F(wk)F(w^k)F(wk) is part of each conclusion, so a junk-zero integral cannot satisfy it.

Two corrections to the paper, each labelled in the item titles and notes:

  • Iterate-wise γ\gammaγ. Assumption 1(3) says Eξ∥∇f(w,ξ)∥2≤γ2E_\xi\|\nabla f(w,\xi)\|^2\le\gamma^2Eξ​∥∇f(w,ξ)∥2≤γ2 for all www. Together with unbiasedness and λ\lambdaλ-strong convexity on Rn\mathbb R^nRn, this forces λ∥w−w∗∥≤∥∇F(w)∥≤γ\lambda\|w-w^*\|\le\|\nabla F(w)\|\le\gammaλ∥w−w∗∥≤∥∇F(w)∥≤γ for every www, which is impossible. The mission imposes the bound at the iterates, conditionally on the past, which is how the proof uses it.
  • Constant Qc(β)Q_c(\beta)Qc​(β). The induction after (3.22) multiplies by 1−2βμ1λ/k1-2\beta\mu_1\lambda/k1−2βμ1​λ/k, which is negative for k<2βμ1λk<2\beta\mu_1\lambdak<2βμ1​λ. Counterexample: n=1n=1n=1, f1,2(w)=(w∓1)2/2f_{1,2}(w)=(w\mp1)^2/2f1,2​(w)=(w∓1)2/2, Hk=IH_k=IHk​=I, w1=0w^1=0w1=0, β=2\beta=2β=2, γ2=5\gamma^2=5γ2=5. Then F(w2)−F(w∗)=2>Q(2)/2=5/3F(w^2)-F(w^*)=2>Q(2)/2=5/3F(w2)−F(w∗)=2>Q(2)/2=5/3, and the example survives perturbing the constants so that every strict inequality holds. The middle entry of QcQ_cQc​ repairs it, and Qc=QQ_c=QQc​=Q whenever 2μ1λβ≤3/22\mu_1\lambda\beta\le3/22μ1​λβ≤3/2.

Smaller conventions:

  • The pairs must satisfy st≠0s_t\neq0st​=0, since Algorithm 2 is undefined otherwise.
  • Hessian samples have size bH≥1b_H\ge1bH​≥1; the empty sample makes (2.3) a 0/00/00/0.
  • The corollary's undefined μ2\mu_2μ2​ is Lemma 3.1's constant, enlarged together with μ1\mu_1μ1​ to cover the initial H=IH=IH=I steps.
  • The block average of Algorithm 1 is used, not Eq. (2.1).

Trivializing encodings are ruled out: HtH_tHt​ is Algorithm 2 applied to Algorithm 1's own pairs, not an arbitrary bounded matrix, and the second-moment bound is not imposed for all www.

Needed infrastructure, all welcome as contributions: the symmetry and spectral bounds of Hessians of C2C^2C2 functions, trace and determinant identities for BFGS updates, the descent lemma from a Hessian upper bound, conditional-expectation manipulations for adapted iterations, and the reduction of Algorithm 1 with uniform finite sampling to the abstract iteration.

Selected references

  • R. H. Byrd, S. L. Hansen, J. Nocedal, Y. Singer, A Stochastic Quasi-Newton Method for Large-Scale Optimization, SIAM J. Optim. 26(2):1008–1031, 2016. https://doi.org/10.1137/140954362
  • A. Mokhtari, A. Ribeiro, Global Convergence of Online Limited Memory BFGS, J. Mach. Learn. Res. 16:3151–3181, 2015. https://jmlr.org/papers/v16/mokhtari15a.html
  • L. Bottou, F. E. Curtis, J. Nocedal, Optimization Methods for Large-Scale Machine Learning, SIAM Review 60(2):223–311, 2018. https://doi.org/10.1137/16M1080173
  • A. Nemirovski, A. Juditsky, G. Lan, A. Shapiro, Robust Stochastic Approximation Approach to Stochastic Programming, SIAM J. Optim. 19(4):1574–1609, 2009. https://doi.org/10.1137/070704277
  • M. J. D. Powell, Some global convergence properties of a variable metric algorithm for minimization without exact line searches, in Nonlinear Programming, SIAM-AMS Proc. IX, 1976, pp. 53–72.
13 thms2 active usersReviewed
🏆Completed
Dynamic ProgrammingOptimizationProbability·Captain: Shuze Chen

Markov Decision Processes XII: Terminal Wealth and Mean-Variance under Partial ObservationTextbook

Motivation

Every portfolio-choice model treated so far in this series assumes the investor knows the exact law governing the market's returns. Real investors do not: the drift of a stock, the regime a market is in, or the probability of an up-move in a simplified binomial model is itself uncertain and must be learned from the very prices being observed. Bäuerle and Rieder's Chapter 6 (Markov Decision Processes with Applications to Finance, Springer, 2011) is the book's synthesis of two threads developed separately earlier: Chapter 5's reduction of a partially observable decision problem to an ordinary one on an enlarged "belief" state space, and Chapter 4's classical solutions of terminal-wealth utility maximization and dynamic mean-variance portfolio choice. Combining them answers a natural question with no simple a priori answer: how does not knowing which market you are in change the qualitatively optimal way to invest, and can the closed-form solutions of the fully-observed theory be recovered, term for term, once the unknown factor is replaced by a belief about it?

Setting

The market has an unobservable factor Y (state space E_Y) driving the vector of relative risks Z ∈ ℝ^d of d risky assets: given Y_n = y, the next return Z_{n+1} has a density q_R(y,\cdot), and Y itself evolves via its own transition density q_Y(y,\cdot), jointly — crucially, this joint law depends only on y, never on wealth or the action taken. An investor observes only the stock prices (equivalently, the return history), never Y itself. Bayes' rule turns this into a filtering problem: the investor's belief ρ_n \in \mathbb P(E_Y) about the current factor is updated one return at a time by an operator Φ(ρ,z) that depends only on the current belief and the newly observed return — a genuine simplification of Chapter 5's general Bayes operator, forced by the market's own structure. The pair (x_n,ρ_n) — observable wealth and current belief — is then an ordinary, fully observed state for an ordinary Markov Decision Model, and every value function and optimal policy of this chapter lives on that enlarged state space.

Formalization targets

The goal, Theorem 6.2.3, solves the dynamic mean-variance problem (MV): minimize the variance of terminal wealth X_N subject to a target expectation \mathbb E[X_N] \ge \mu, under partial observation. It is reached by a Lagrangian embedding into an auxiliary quadratic-loss problem QP(b), solved explicitly in Theorem 6.2.2, whose value function factors as ((xS^0_N/S^0_n)-b)^2 d_n(\rho) for a belief-only sequence (d_n) satisfying the backward recursion (6.7); Lemma 6.2.1 shows this sequence always lies strictly between 0 and 1, which is exactly what makes the final variance formula and Lagrange multiplier well-posed. The remaining milestones develop the parallel terminal-wealth theory of §6.1: the general structure theorem (Theorem 6.1.1), its power- and logarithmic-utility closed forms (Theorems 6.1.2, 6.1.7), and — for the specific binomial market with an unknown up-probability — a likelihood-ratio monotonicity result for the filter update (Lemma 6.1.4) and a comparison between the partially and completely observed optimal investment fractions (Theorem 6.1.5).

Significance

The chapter's organizing insight is that partial observation does not require a new theory: once the belief is added as a state coordinate, every general result already proved for fully observed Markov Decision Models — the Bellman equation, the existence of optimal Markov policies, the Lagrangian embedding technique for mean-variance problems — applies unchanged. What is genuinely new, and genuinely non-trivial, is checking that the reduced model inherits the structural hypotheses (monotonicity, boundedness, positive-definiteness of covariance matrices) those general theorems require, expressed now as conditions on the belief-indexed quantities Φ(ρ,z), d_n(\rho), \ell_n(\rho), C_n(\rho) rather than on the original, unobserved factor. Theorem 6.1.5's comparison result is a genuinely new phenomenon with no fully-observed analogue at all: it quantifies, in the two opposite directions dictated by the sign of the risk-aversion parameter γ, how residual uncertainty about the market itself changes the qualitatively optimal amount to invest — the discrete-time analogue of a continuous-time result in the literature this book cites (Sass and Haussmann 2004).

Difficulty

The recurring difficulty across every result in this mission is that the reduced model's state space E_X \times \mathbb P(E_Y) includes a space of probability measures as one coordinate, and every quantity that must be shown well-defined, monotone, or bounded is a functional on that space, not a function on a concrete Euclidean set. Formalizing the mean-variance recursion (6.7) in particular is a three-way mutual computation — a scalar d_n(\rho), a vector \ell_n(\rho), and a matrix C_n(\rho), each an integral against the same belief-dependent predictive law of the next return, each feeding the next stage's version of all three — where Lemma 6.2.1's strict-inequality bound is not a bookkeeping detail but exactly the fact that keeps C_n(\rho) invertible and the whole construction from breaking down. Theorem 6.2.3 itself is the hardest single step: verifying that the specific constant b^* the Lagrangian method selects makes the mean constraint bind at exact equality, and that the resulting variance is the true constrained minimum (not merely a feasible value), is exactly the non-trivial content a superficial restatement of Theorem 6.2.2 at an unspecified b would silently discard.

Formalization scope

Every value function of this chapter — the terminal-wealth maximization of §6.1, the quadratic loss QP(b) and the mean-variance problem (MV) of §6.2 — is built from one shared history-dependent value-function scaffold, parametrized by its terminal payoff (the utility U, a quadratic loss, or the raw first/second moment), its rate sequence (constant in §6.1, non-stationary in §6.2), and its feasible-action correspondence, rather than four separately re-derived constructions. Optimal fractions in the binomial sub-model (Lemma 6.1.4, Theorem 6.1.5) are characterized as any maximizer of the relevant one-step concave problem rather than through the closed-form solution the book's own proof derives via machinery from a different, unavailable chunk (Lemma 4.2.9) — the comparison and monotonicity results proved here are facts about any such maximizer, not about that specific formula. A formalization that assumed the reduced model's filter update or covariance structure directly, rather than deriving it from the market's own return and factor densities via the Bayes operator Φ, would trivialize every result in this mission; none of the items here take that shortcut.

Selected references

  • N. Bäuerle and U. Rieder, Markov Decision Processes with Applications to Finance, Universitext, Springer, 2011. DOI: 10.1007/978-3-642-18324-9.
  • R. Sass and U. G. Haussmann, "Optimizing trading strategies with respect to drawdown in the hidden Markov model," Statistics & Decisions, 2004.
  • N. Bäuerle and U. Rieder, "Portfolio optimization with unobservable Markov-modulated drift process," Journal of Applied Probability, 2007.
  • V. Runggaldier, W. Trivellato, and T. Vargiolu, "A Bayesian adaptive control approach to parameter estimation and optimal portfolio selection," in Mathematical Finance, Trends in Mathematics, Birkhäuser, 2002 (the binomial-market source this chapter's §6.1 example specializes).
14 thms2 active usersReviewed
PreviousPage 28 of 44Next

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paper
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me