Prove2Me
Navigate
DiscoverCollectionsFormalpediaBlogsUsersMomentumMy Missions+
Prove2Me
⌕
Log in
← Collections

The OR Formalization Drive

Help us formalize the operations research literature in Lean.

1094 missions

Missions

761–780 of 1094
OpenCompletedAll
🏆Completed
CombinatoricsOperations ResearchTheoretical Computer Science·Captain: mikedeng1

The Online Set Cover Problem 2: Given α ≥ c(C_OPT), the Weighted Potential-Function Algorithm Never Fails and Pays at Most (6+o(1)) α log m log nResearch Paper

Motivation

Set cover is one of the basic covering problems of combinatorial optimization: given a ground set and a family of subsets with costs, choose a cheapest subfamily whose union contains every element. In many applications the elements to be covered are not known in advance but appear over time: requests for a service that must be served by opening facilities, clients that must be assigned to servers, or constraints of a covering program that are revealed one at a time. Each arriving element must be covered at once, and decisions cannot be undone. This is the online set cover problem, introduced by Alon, Awerbuch, Azar, Buchbinder and Naor (SIAM J. Comput. 39(2), 2009; conference version STOC 2003).

The quality of an online algorithm is measured by its competitive ratio: the worst case, over all arrival sequences, of the ratio between the algorithm's cost and the cost of an optimal offline cover of the elements that actually arrived. The paper gives a deterministic algorithm with ratio O(log⁡mlog⁡n)O(\log m \log n)O(logmlogn), where nnn is the number of elements and mmm the number of sets, and shows a nearly matching lower bound for deterministic algorithms. Its algorithm for the weighted case, analysed with a potential function, became a template for the online primal–dual method surveyed by Buchbinder and Naor (Found. Trends Theor. Comput. Sci. 3(2–3), 2009).

This mission formalizes the core of the weighted result: the algorithm that is given a value α\alphaα at least the optimal cost, and its guarantee (Theorem 3.4).

Setting

The ground set XXX has n=∣X∣n = |X|n=∣X∣ elements and the family S\mathcal SS has m=∣S∣m = |\mathcal S|m=∣S∣ sets; every set SSS has a cost cS>0c_S > 0cS​>0. Both are known to the algorithm in advance. For an element jjj, Sj\mathcal S_jSj​ denotes the sets containing jjj. Elements of an unknown subset of XXX arrive one at a time in a sequence σ\sigmaσ; on arrival each must be covered by a chosen set. The chosen family C\mathcal CC can only grow. COPT\mathcal C_{OPT}COPT​ is any family covering every arriving element, and c(COPT)=∑S∈COPTcSc(\mathcal C_{OPT}) = \sum_{S \in \mathcal C_{OPT}} c_Sc(COPT​)=∑S∈COPT​​cS​.

The algorithm is given α≥c(COPT)\alpha \ge c(\mathcal C_{OPT})α≥c(COPT​). It discards sets costing more than α\alphaα, buys sets costing at most α/m\alpha/mα/m outright, and rescales costs; on the resulting normalized instance 1≤cS≤m1 \le c_S \le m1≤cS​≤m and cS≤αc_S \le \alphacS​≤α for every set (p. 365).

The algorithm keeps a weight wS>0w_S > 0wS​>0 for every set, initially wS=1/m2w_S = 1/m^2wS​=1/m2; the weight of an element is wj=∑S∈SjwSw_j = \sum_{S \in \mathcal S_j} w_Swj​=∑S∈Sj​​wS​. With CCC the set of covered elements and χC\chi_{\mathcal C}χC​ the indicator of C\mathcal CC, the potential is

Φ=∑j∉Cn2wj+n⋅exp⁡(12α∑S∈S(cSχC(S)−3wScSlog⁡n)),\Phi = \sum_{j \notin C} n^{2 w_j} + n \cdot \exp\Big(\frac{1}{2\alpha} \sum_{S \in \mathcal S} \big(c_S \chi_{\mathcal C}(S) - 3 w_S c_S \log n\big)\Big),Φ=j∈/C∑​n2wj​+n⋅exp(2α1​S∈S∑​(cS​χC​(S)−3wS​cS​logn)),

with natural logarithms throughout. When jjj arrives with wj≥1w_j \ge 1wj​≥1 nothing happens; otherwise the algorithm performs weight augmentation steps while wj<1w_j < 1wj​<1. In a step, for each S∈SjS \in \mathcal S_jS∈Sj​: (a) wS←wS(1+1ncS)w_S \leftarrow w_S (1 + \frac{1}{n c_S})wS​←wS​(1+ncS​1​); (b) if S∉CS \notin \mathcal CS∈/C, add SSS to C\mathcal CC when Φ\PhiΦ does not exceed its value before (a); (c) if Φ\PhiΦ has increased, return FAIL.

In Lean, the instance is the published OnlinePrimalDual.OnlineSetCover.SetCoverInstance over finite types X (elements) and T (sets), with the published elementWeight, coveredBy and potential. The run is OnlineSetCover.Weighted.Reachable inst α σ, the set of configurations reachable from initState σ under the transition relation Step.

Formalization targets

Goal: Theorem 3.4

On the normalized instance, with COPT\mathcal C_{OPT}COPT​ covering σ\sigmaσ, c(COPT)≤αc(\mathcal C_{OPT}) \le \alphac(COPT​)≤α, and n⋅n2/m+n<n2n \cdot n^{2/m} + n < n^2n⋅n2/m+n<n2, every reachable configuration is a running state (never FAIL) in which (i) every j∈Xj \in Xj∈X with wj≥1w_j \ge 1wj​≥1 is covered, and (ii)

∑S∈CcS≤3log⁡n(1+(1+1n)αlog⁡(m2(1+1n)))+2αlog⁡n=(6+o(1)) αlog⁡mlog⁡n.\sum_{S \in \mathcal C} c_S \le 3 \log n \Big(1 + \Big(1 + \frac1n\Big)\alpha \log\Big(m^2\Big(1+\frac1n\Big)\Big)\Big) + 2\alpha \log n = (6 + o(1))\,\alpha \log m \log n.S∈C∑​cS​≤3logn(1+(1+n1​)αlog(m2(1+n1​)))+2αlogn=(6+o(1))αlogmlogn.

Milestones

  • Lemma 3.1 (p. 365): the number NNN of augmentation steps satisfies N≤∑S∈COPT(ncS+1)log⁡(m2(1+1/n))≤(n+1)αlog⁡(m2(1+1/n))N \le \sum_{S \in \mathcal C_{OPT}} (n c_S + 1)\log(m^2(1 + 1/n)) \le (n+1)\alpha\log(m^2(1+1/n))N≤∑S∈COPT​​(ncS​+1)log(m2(1+1/n))≤(n+1)αlog(m2(1+1/n)).
  • Lemma 3.2 (p. 366): throughout, ∑SwScS≤1+N/n≤1+(1+1/n)αlog⁡(m2(1+1/n))\sum_S w_S c_S \le 1 + N/n \le 1 + (1 + 1/n)\alpha\log(m^2(1+1/n))∑S​wS​cS​≤1+N/n≤1+(1+1/n)αlog(m2(1+1/n)).
  • Lemma 3.3 (p. 366): a per-set step with cS≤αc_S \le \alphacS​≤α never increases Φ\PhiΦ; in particular the algorithm never fails.

The Proved platform theorem OnlinePrimalDual.OnlineSetCover.algorithm_correctness (the last paragraph of the proof of Theorem 3.4, with the invariant Φ<n2\Phi < n^2Φ<n2 assumed) is included as a supporting reference.

Significance

Theorem 3.4 is the analysis of the subroutine; with the doubling over guesses of α\alphaα described on pp. 364–365 (which loses a factor of at most 4) it yields the paper's deterministic O(log⁡mlog⁡n)O(\log m \log n)O(logmlogn)-competitive algorithm for weighted online set cover. The lower bound of Section 4 shows that no deterministic algorithm can do much better on general instances, so the result is close to the deterministic optimum. The technique, a potential that couples a fractional multiplicative-weights solution to a deterministic rounding, reappears in online covering and packing, online facility location and related problems.

The result is proved in the paper and restated in the Buchbinder–Naor monograph. On Prove2Me, the monograph's final step (from the invariant Φ<n2\Phi < n^2Φ<n2 to the cost bound) is a Proved theorem, and its expectation form of the monotonicity lemma is Disproved because it omits the hypothesis cS≤αc_S \le \alphacS​≤α. Neither the full statement about the algorithm's run nor Lemmas 3.1, 3.2 and the corrected Lemma 3.3 are formalized on the platform. This mission produces them, with the o(1)o(1)o(1) terms replaced by explicit expressions.

Difficulty

The cost bound in the last step is short once two facts about the run are available: that Φ\PhiΦ stays below n2n^2n2, and that the fractional cost ∑SwScS\sum_S w_S c_S∑S​wS​cS​ stays logarithmic. Neither is a local fact about one state. The first requires showing that, at every per-set step, one of the two deterministic choices (add SSS or not) does not increase Φ\PhiΦ; the paper proves this by a probabilistic argument over an auxiliary randomized choice, and the bound on the exponential term depends on the cost of the set being at most α\alphaα. The platform's earlier statement of this lemma, which omits that hypothesis, is Disproved. The second requires a bound on the number of augmentation steps over the whole run, which depends on the run's history and not on any single state. In Lean both are inductions over an operational semantics with real-valued exponentials and powers n2wjn^{2 w_j}n2wj​, where the initial bound Φ<n2\Phi < n^2Φ<n2 is a genuine size condition on nnn and mmm.

Formalization scope

The run is a small-step transition relation. A state records the weights, the cover, the number of augmentation steps begun, the elements not yet given, and the position inside the current step; FAIL is a separate terminal configuration. The order in which a step visits Sj\mathcal S_jSj​ is arbitrary and may differ between steps; every statement holds for every order. "Throughout the algorithm" means every reachable configuration, including those between per-set substeps. Arrival sequences are arbitrary lists (repetitions allowed) of elements covered by COPT\mathcal C_{OPT}COPT​.

Conventions: costs, weights and α\alphaα are real; nnn and mmm are the cardinalities of the finite types cast to R\mathbb RR; log⁡\loglog is Real.log; n2wjn^{2 w_j}n2wj​ and n2/mn^{2/m}n2/m are real powers. The paper's asymptotic expressions are replaced by what its proofs establish:

  • Lemma 3.1: (2+o(1))nαlog⁡m(2 + o(1)) n\alpha\log m(2+o(1))nαlogm becomes (n+1)αlog⁡(m2(1+1/n))(n+1)\alpha\log(m^2(1+1/n))(n+1)αlog(m2(1+1/n));
  • Lemma 3.2: (2+o(1))αlog⁡m(2 + o(1))\alpha\log m(2+o(1))αlogm becomes 1+(1+1/n)αlog⁡(m2(1+1/n))1 + (1+1/n)\alpha\log(m^2(1+1/n))1+(1+1/n)αlog(m2(1+1/n)), together with the intermediate bound 1+N/n1 + N/n1+N/n;
  • Theorem 3.4 (ii): (6+o(1))αlog⁡mlog⁡n(6 + o(1))\alpha\log m\log n(6+o(1))αlogmlogn becomes 3log⁡n (1+(1+1/n)αlog⁡(m2(1+1/n)))+2αlog⁡n3\log n\,(1 + (1+1/n)\alpha\log(m^2(1+1/n))) + 2\alpha\log n3logn(1+(1+1/n)αlog(m2(1+1/n)))+2αlogn;
  • "n and m large" becomes the hypothesis n⋅n2/m+n<n2n \cdot n^{2/m} + n < n^2n⋅n2/m+n<n2 used for the initial potential (it holds, for instance, when n≥4n \ge 4n≥4 and m≥3m \ge 3m≥3).

The goal is a statement about the configurations the algorithm actually reaches from wS=1/m2w_S = 1/m^2wS​=1/m2 and the empty cover. Taking the invariant Φ<n2\Phi < n^2Φ<n2 or the fractional-cost bound as a hypothesis on an arbitrary state would trivialize it, and is ruled out: those are exactly what the milestones establish. The doubling wrapper for unknown α\alphaα is not part of this mission.

A complete development needs an invariant for reachable states (positive weights, steps of an element processed in full), the per-set potential inequality, and the step-counting argument. The per-set inequality is reusable for the monograph's version of the algorithm. Contributions of proofs of any milestone, and of auxiliary invariants as separate lemmas, are welcome.

Selected references

  • N. Alon, B. Awerbuch, Y. Azar, N. Buchbinder, J. Naor, The Online Set Cover Problem, SIAM Journal on Computing 39(2):361–370, 2009. https://doi.org/10.1137/060661946
  • N. Buchbinder, J. Naor, The Design of Competitive Online Algorithms via a Primal–Dual Approach, Foundations and Trends in Theoretical Computer Science 3(2–3):93–263, 2009. https://doi.org/10.1561/0400000024
10 thms2 active usersReviewed
Bandit AlgorithmsMachine LearningOperations Research+1·Captain: mikedeng1

Analysis of Thompson Sampling for the Multi-armed Bandit Problem 2: Logarithmic Regret for N ArmsResearch Paper

Motivation

Thompson Sampling is the oldest heuristic for the multi-armed bandit problem: proposed by Thompson in 1933, it plays each arm with the posterior probability that the arm is the best one. It is simple to implement, performs well empirically (Chapelle and Li, NIPS 2011), and has been used in production systems such as click-through-rate prediction for search advertising. For a long time, however, no finite-time regret guarantee was known for it: the analyses available before 2012 gave only o(T)o(T)o(T) regret.

Agrawal and Goyal (arXiv:1111.1797, COLT 2012) gave the first logarithmic bounds on the expected regret of Thompson Sampling. This mission formalizes their bound for the general case of NNN arms (their Theorem 2). A companion mission of the same series formalizes their two-armed bound (Theorem 1), whose proof is independent.

Timeline. Lai and Robbins (1985) proved that every consistent algorithm has regret at least of order ∑iΔiD(μi∥μ1)ln⁡T\sum_i \frac{\Delta_i}{D(\mu_i\|\mu_1)}\ln T∑i​D(μi​∥μ1​)Δi​​lnT. Auer, Cesa-Bianchi and Fischer (2002) showed that UCB1 achieves O(∑iln⁡T/Δi)O(\sum_i \ln T/\Delta_i)O(∑i​lnT/Δi​) in finite time. Agrawal and Goyal (2012) proved O((∑a1/Δa2)2ln⁡T)O((\sum_a 1/\Delta_a^2)^2\ln T)O((∑a​1/Δa2​)2lnT) for Thompson Sampling with NNN arms; Kaufmann, Korda and Munos (2012) and Agrawal and Goyal (2013) later proved the asymptotically optimal constant for Bernoulli rewards.

Setting

A stochastic NNN-armed bandit has arms 1,…,N1,\dots,N1,…,N. Arm iii, when played, yields a random reward drawn from a fixed distribution νi\nu_iνi​ supported in [0,1][0,1][0,1], with mean μi\mu_iμi​; rewards of an arm are i.i.d. and independent of the other arms. Arm 111 is assumed to be the unique optimal arm, μ1>μi\mu_1>\mu_iμ1​>μi​ for i≠1i\ne1i=1, and Δi=μ1−μi>0\Delta_i=\mu_1-\mu_i>0Δi​=μ1​−μi​>0 is the gap of arm iii.

Thompson Sampling for general stochastic bandits (Algorithm 2 of the paper) keeps, for each arm iii, a count SiS_iSi​ of successes and FiF_iFi​ of failures, both starting at 000. In each round ttt it draws θi(t)∼Beta(Si+1,Fi+1)\theta_i(t)\sim\mathrm{Beta}(S_i+1,F_i+1)θi​(t)∼Beta(Si​+1,Fi​+1) independently for every arm, plays i(t)=arg⁡max⁡iθi(t)i(t)=\arg\max_i\theta_i(t)i(t)=argmaxi​θi​(t), observes a reward r~t∼νi(t)\tilde r_t\sim\nu_{i(t)}r~t​∼νi(t)​, performs a Bernoulli trial with success probability r~t\tilde r_tr~t​, and increments Si(t)S_{i(t)}Si(t)​ on success and Fi(t)F_{i(t)}Fi(t)​ on failure.

The expected regret in time TTT is

E[R(T)]=E[∑t=1T(μ∗−μi(t))],μ∗=max⁡iμi,\mathbb E[\mathcal R(T)]=\mathbb E\Big[\sum_{t=1}^T(\mu^*-\mu_{i(t)})\Big],\qquad \mu^*=\max_i\mu_i,E[R(T)]=E[t=1∑T​(μ∗−μi(t)​)],μ∗=imax​μi​,

the expectation being over the rewards, the Bernoulli trials and the posterior samples.

The proof works with the reward stacks Zi,mZ_{i,m}Zi,m​: the outcome of the mmm-th Bernoulli trial of arm iii, all independent. Then s(j)=∑m≤jZ1,ms(j)=\sum_{m\le j}Z_{1,m}s(j)=∑m≤j​Z1,m​, the number of successes in the first jjj plays of arm 111, is a Binomial(j,μ1)\mathrm{Binomial}(j,\mu_1)Binomial(j,μ1​) random variable. The other objects of the proof are the threshold Li=24ln⁡T/Δi2L_i=24\ln T/\Delta_i^2Li​=24lnT/Δi2​, the saturated set C(t)C(t)C(t) of suboptimal arms with at least LiL_iLi​ plays before round ttt, the intervals IjI_jIj​ between the jjj-th and (j+1)(j+1)(j+1)-th plays of arm 111, and the counts γj\gamma_jγj​ and Vjℓ,aV_j^{\ell,a}Vjℓ,a​ defined in §4.

Formalization targets

Goal: Theorem 2

There is an absolute constant C>0C>0C>0 such that for every N≥2N\ge2N≥2, every instance as above and every horizon T≥2T\ge2T≥2,

E[R(T)]≤C(∑a=2N1Δa2)2ln⁡T.\mathbb E[\mathcal R(T)]\le C\Big(\sum_{a=2}^N\frac{1}{\Delta_a^2}\Big)^2\ln T .E[R(T)]≤C(a=2∑N​Δa2​1​)2lnT.

CCC does not depend on NNN, on the reward distributions or on TTT.

Milestones

  1. Lemma 4: with E(t)E(t)E(t) the event that every saturated arm's sample lies within Δi/2\Delta_i/2Δi​/2 of its mean, Pr⁡(E(t))≥1−4(N−1)/T2\Pr(E(t))\ge1-4(N-1)/T^2Pr(E(t))≥1−4(N−1)/T2, also conditionally on s(j)=ss(j)=ss(j)=s.
  2. Lemma 5 (Eq. (7)): the expected regret from saturated arms inside IjI_jIj​ is at most E[E[γj+1∣s(j)]∑aΔaE[min⁡{X(j,s(j),μa+Δa/2),T}∣s(j)]]\mathbb E\big[\mathbb E[\gamma_j+1\mid s(j)]\sum_a\Delta_a\mathbb E[\min\{X(j,s(j),\mu_a+\Delta_a/2),T\}\mid s(j)]\big]E[E[γj​+1∣s(j)]∑a​Δa​E[min{X(j,s(j),μa​+Δa​/2),T}∣s(j)]].
  3. Lemma 1: E[X(j,s,y)]=1/Fj+1,yB(s)−1\mathbb E[X(j,s,y)]=1/F^B_{j+1,y}(s)-1E[X(j,s,y)]=1/Fj+1,yB​(s)−1, where X(j,s,y)X(j,s,y)X(j,s,y) counts the trials before an independent Beta(s+1,j−s+1)\mathrm{Beta}(s+1,j-s+1)Beta(s+1,j−s+1) sample exceeds yyy.
  4. Lemma 3: a three-case bound on E[E[min⁡{X(j,s(j),y),T}∣s(j)]]\mathbb E[\mathbb E[\min\{X(j,s(j),y),T\}\mid s(j)]]E[E[min{X(j,s(j),y),T}∣s(j)]] in terms of the Bernoulli KL divergence DDD between yyy and μ1\mu_1μ1​.

Significance

The result. Theorem 2 shows that Thompson Sampling, a randomized Bayesian heuristic, achieves regret logarithmic in the horizon for any number of arms with bounded rewards, matching the order in TTT of the Lai–Robbins lower bound. Its dependence on the gaps, (∑aΔa−2)2(\sum_a\Delta_a^{-2})^2(∑a​Δa−2​)2, is worse than UCB1's; the paper's own Remark 1 and later work improve it. The proof introduced the device of bounding the waiting time between plays of the optimal arm through geometric variables with Beta-cdf parameters (Lemmas 1 and 3), which reappears in later analyses of Thompson Sampling.

Formalizing it. The theorem is proved on paper; it has not been machine-checked. Bandit theory in Lean (bandit environments, regret, UCB-type analyses) is still young, and no Beta–Bernoulli Thompson Sampling result is formalized. The mission produces a Lean model of Algorithm 2 for general [0,1][0,1][0,1] rewards with the paper's stack coupling, the §4 bookkeeping of saturated arms and intervals, and the paper's lemmas as separate targets.

Difficulty

Two difficulties are specific to the NNN-armed analysis. First, the arm that competes with arm 111 changes over time: the set of saturated arms grows, and which saturated arm is "best" depends on the history, so the waiting time between plays of arm 111 cannot be compared with a single geometric variable as in the two-armed case. Second, the number γj\gamma_jγj​ of rounds at which arm 111's sample is large but arm 111 is not played is not independent of the counts Vjℓ,aV_j^{\ell,a}Vjℓ,a​: both depend on the same posterior samples, and Lemma 5 needs a careful conditioning on the history to separate them. The obvious union bound over arms, treating each suboptimal arm as in the two-armed proof, fails because it ignores the interruptions by unsaturated arms, whose number is the source of the squared sum in the bound.

Formalization scope

  • Probability space. Algorithm 2 is realized on a product of three independent i.i.d. tables: Beta draws indexed by (arm, round, successes, failures), rewards indexed by (arm, round) and uniform variables indexed by (arm, round); the Bernoulli trial of a round succeeds when the played arm's uniform variable is below its reward. The law of the run is that of Algorithm 2, which runs for every round t=1,2,…t=1,2,\dotst=1,2,…. s(j)s(j)s(j) is the number of successful trials among the first jjj plays of arm 111 in this infinite run (possibly after the horizon TTT), so it is a Binomial(j,μ1)\mathrm{Binomial}(j,\mu_1)Binomial(j,μ1​) random variable for every jjj, as the paper's independent Z1,mZ_{1,m}Z1,m​ make it. Ties in the arg max (probability 000) go to the smallest index.
  • Indexing. Arms are Fin N, and Lean arm 0 is the paper's arm 111. Rounds are 0,…,T−10,\dots,T-10,…,T−1; Lean round ttt is the paper's round t+1t+1t+1. Sums over a=2,…,Na=2,\dots,Na=2,…,N are sums over a≠0a\ne0a=0.
  • Expectations are lower Lebesgue integrals in [0,∞][0,\infty][0,∞], which has no junk value for non-integrable functions. Conditional expectations given s(j)s(j)s(j) are written as finite sums over the values of s(j)s(j)s(j).
  • The O(⋅)O(\cdot)O(⋅). The paper writes O(⋅)O(\cdot)O(⋅) in the sense of its footnote 1 (f(n)≤c g(n)f(n)\le c\,g(n)f(n)≤cg(n) for n≥n0n\ge n_0n≥n0​). The goal states it with one universal constant CCC, quantified before NNN, the instance and TTT, for every T≥2T\ge2T≥2. The explicit constants printed in App. D are not formalized: expanding the paper's Eq. (21) gives terms 288(N−1)(ln⁡T)∑aΔa−2288(N-1)(\ln T)\sum_a\Delta_a^{-2}288(N−1)(lnT)∑a​Δa−2​ and 48(N−1)248(N-1)^248(N−1)2 where the paper prints 288(ln⁡T)∑iΔi−2288(\ln T)\sum_i\Delta_i^{-2}288(lnT)∑i​Δi−2​, and Eq. (22) drops a factor ln⁡T\ln TlnT in its 192/Δa2192/\Delta_a^2192/Δa2​ term. The O(⋅)O(\cdot)O(⋅) claim does not depend on these slips; a statement pinned to the printed numerals might be false.
  • Ruled out. A constant depending on NNN, on the means or on TTT; a fixed number of arms; Bernoulli rewards only; or any algorithm other than Algorithm 2 would each make the goal a different and weaker theorem. The statement quantifies over all N≥2N\ge2N≥2 and all reward distributions on [0,1][0,1][0,1].
  • Not included. Eq. (8), the bound ∑jE[γj∣s(j)]≤∑uLu+4(N−1)\sum_{j}\mathbb E[\gamma_j\mid s(j)]\le\sum_uL_u+4(N-1)∑j​E[γj​∣s(j)]≤∑u​Lu​+4(N−1) "for all instantiations", is not a milestone: each term is conditioned on a different s(j)s(j)s(j), and the pointwise reading does not follow from the argument given. Remark 1 (an alternate bound) and App. A (several optimal arms) are not part of this mission.
  • Contributions welcome: Beta–Binomial identities, geometric waiting times, Hoeffding bounds for binomial cdfs, and the stopping-time arguments behind Lemma 5. Lemma 1 and Lemma 3 are shared with the two-armed mission of this series.

Selected references

  • S. Agrawal and N. Goyal, Analysis of Thompson Sampling for the Multi-armed Bandit Problem, COLT 2012; arXiv:1111.1797v3, 2012. https://arxiv.org/abs/1111.1797
  • W. R. Thompson, On the likelihood that one unknown probability exceeds another in view of the evidence of two samples, Biometrika 25, 1933. https://doi.org/10.1093/biomet/25.3-4.285
  • T. L. Lai and H. Robbins, Asymptotically efficient adaptive allocation rules, Advances in Applied Mathematics 6, 1985. https://doi.org/10.1016/0196-8858(85)90002-8
  • P. Auer, N. Cesa-Bianchi and P. Fischer, Finite-time analysis of the multiarmed bandit problem, Machine Learning 47, 2002. https://doi.org/10.1023/A:1013689704352
  • O. Chapelle and L. Li, An empirical evaluation of Thompson Sampling, NIPS 2011. https://papers.nips.cc/paper/4321-an-empirical-evaluation-of-thompson-sampling
  • E. Kaufmann, N. Korda and R. Munos, Thompson Sampling: an asymptotically optimal finite-time analysis, ALT 2012. https://arxiv.org/abs/1205.4217
  • S. Agrawal and N. Goyal, Further optimal regret bounds for Thompson Sampling, AISTATS 2013. https://arxiv.org/abs/1209.3353
11 thms3 active usersReviewed
🏆Completed
Algorithmic Game TheoryOperations ResearchProbability·Captain: mikedeng1

The Price of Anarchy of Finite Congestion Games V: The Mixed Price of Anarchy of the Average Social Cost Is at Most (3+sqrt 5)/2Research Paper

Motivation

When many self-interested users share resources whose cost grows with use (links of a network, servers, machines), each user picks the option that is cheapest for them given what the others do, and the resulting equilibrium can be worse for the group than a centrally planned allocation. The price of anarchy, introduced by Koutsoupias and Papadimitriou (STACS 1999), measures this loss as the worst ratio between the social cost of an equilibrium and the optimal social cost. Congestion games (Rosenthal 1973) are the standard finite model of such resource sharing: they always have pure Nash equilibria, and they cover atomic routing, load balancing and many network design questions.

Timeline of the linear-latency case:

  • 2002: Roughgarden and Tardos (J. ACM) bound the price of anarchy of nonatomic selfish routing with linear latencies by 4/34/34/3.
  • 2005: Awerbuch, Azar and Epstein (STOC 2005) and, independently, Christodoulou and Koutsoupias (STOC 2005) show that for atomic (finite) congestion games with linear latencies the pure price of anarchy of the total cost is 5/25/25/2. Awerbuch, Azar and Epstein also obtain (3+5)/2≈2.618(3+\sqrt5)/2 \approx 2.618(3+5​)/2≈2.618 for weighted players.
  • 2005: Christodoulou and Koutsoupias observe that their argument for 5/25/25/2 extends to mixed Nash equilibria, at the price of the larger constant (3+5)/2(3+\sqrt5)/2(3+5​)/2 (their Theorem 14, the goal of this mission).

Setting

A congestion game consists of a finite set NNN of players, a finite set EEE of facilities, for each player iii a collection Σi\Sigma_iΣi​ of pure strategies, each a subset of EEE, and for each facility eee a latency function fe:N→Rf_e : \mathbb N \to \mathbb Rfe​:N→R. In a pure profile A=(A1,…,An)A = (A_1, \dots, A_n)A=(A1​,…,An​), Ai∈ΣiA_i \in \Sigma_iAi​∈Σi​, the load ne(A)n_e(A)ne​(A) is the number of players whose set contains eee, and player iii pays

ci(A)=∑e∈Aife(ne(A)).c_i(A) = \sum_{e \in A_i} f_e\bigl(n_e(A)\bigr).ci​(A)=e∈Ai​∑​fe​(ne​(A)).

The social cost of a pure profile is SUM(A)=∑i∈Nci(A)\mathrm{SUM}(A) = \sum_{i\in N} c_i(A)SUM(A)=∑i∈N​ci​(A) (NNN times the average cost). Latencies are linear: fe(k)=aek+bef_e(k) = a_e k + b_efe​(k)=ae​k+be​ with ae,be≥0a_e, b_e \ge 0ae​,be​≥0.

A mixed strategy pip_ipi​ of player iii is a probability distribution on Σi\Sigma_iΣi​. The players randomize independently, so the pure profile sss occurs with probability Pr⁡(s)=∏jpj(sj)\Pr(s) = \prod_j p_j(s_j)Pr(s)=∏j​pj​(sj​). Player iii's expected cost is E[ci]=∑sPr⁡(s) ci(s)\mathbb E[c_i] = \sum_s \Pr(s)\, c_i(s)E[ci​]=∑s​Pr(s)ci​(s), and the expected load of eee is E[ne]=∑sPr⁡(s) ne(s)\mathbb E[n_e] = \sum_s \Pr(s)\, n_e(s)E[ne​]=∑s​Pr(s)ne​(s). A mixed profile is a mixed Nash equilibrium if no player can lower their expected cost by switching unilaterally to another distribution on their strategies. The social cost of a mixed profile is the sum of the expected costs, SUM(p)=∑iE[ci]\mathrm{SUM}(p) = \sum_i \mathbb E[c_i]SUM(p)=∑i​E[ci​].

In Lean: CongestionGame ι E with fields strategies and latency, load, cost, IsProfile, sumCost, IsLinear, expCost, expLoad, mixedSumCost and IsMixedNash, all in the namespace CongestionPoA.Mixed; lotteries and independent randomization come from the platform definition agt_games (AGT.IsLottery, AGT.profileProb, AGT.IsMixedNash).

Formalization targets

Goal: Theorem 14

For every congestion game with linear latencies, every mixed Nash equilibrium ppp and every pure profile PPP with Pi∈ΣiP_i \in \Sigma_iPi​∈Σi​,

∑i∈NE[ci]  ≤  3+52 SUM(P).\sum_{i\in N} \mathbb E[c_i] \;\le\; \frac{3+\sqrt5}{2}\,\mathrm{SUM}(P).i∈N∑​E[ci​]≤23+5​​SUM(P).

Taking PPP optimal, the mixed price of anarchy of the average social cost is at most (3+5)/2(3+\sqrt5)/2(3+5​)/2.

Milestones

  1. Lemma 3 (corrected). For real x≥0x \ge 0x≥0 and integer y≥0y \ge 0y≥0, y(x+1)≤5−14x2+5+54y2y(x+1) \le \frac{\sqrt5-1}{4}x^2 + \frac{\sqrt5+5}{4}y^2y(x+1)≤45​−1​x2+45​+5​y2.
  2. Deviation inequality (Theorem 1's proof, mixed). At a mixed Nash equilibrium, for every player iii,
E[ci]≤∑e∈Pi(ae(E[ne]+1)+be).\mathbb E[c_i] \le \sum_{e\in P_i}\bigl(a_e(\mathbb E[n_e]+1) + b_e\bigr).E[ci​]≤e∈Pi​∑​(ae​(E[ne​]+1)+be​).
  1. Summing step (Theorem 1's proof, mixed).
∑iE[ci]≤∑e∈Ene(P)(ae(E[ne]+1)+be).\sum_i \mathbb E[c_i] \le \sum_{e\in E} n_e(P)\bigl(a_e(\mathbb E[n_e]+1) + b_e\bigr).i∑​E[ci​]≤e∈E∑​ne​(P)(ae​(E[ne​]+1)+be​).

Significance

The bound says that randomization by the players cannot make linear congestion games much worse than pure play: the loss stays within a constant factor independent of the number of players and facilities. Mixed equilibria matter because they always exist in every finite game and because they model populations of users whose individual choices are not known in advance. The same authors report (PDF p. 2) extending these bounds to correlated equilibria with the same values, which places the mixed bound in a hierarchy of equilibrium notions whose worst cases coincide.

The result is proved in the paper, in one sentence that defers to the proof of Theorem 1. The work here is a complete machine-checked version: the expected-cost layer for congestion games on top of agt_games, the deviation and summing inequalities under product distributions, and the corrected Lemma 3. The paper prints Lemma 3 for all nonnegative reals x,yx, yx,y, where it is false (at x=0x = 0x=0, y=1/10y = 1/10y=1/10 the left side is 0.10.10.1 and the right side about 0.0180.0180.018); the formal statement keeps yyy an integer, which is how the lemma is used. No formalization of congestion-game price-of-anarchy bounds was on the platform when this mission was drafted.

Difficulty

The pure proof compares ne(P)(ne(A)+1)n_e(P)(n_e(A)+1)ne​(P)(ne​(A)+1) with ne(A)2n_e(A)^2ne​(A)2 and ne(P)2n_e(P)^2ne​(P)2 through an integer inequality (Lemma 1). Under a mixed equilibrium the equilibrium side is an expectation, so the cost of a facility is no longer a function of one integer load, and Lemma 1's constant 1/31/31/3 is not available for real arguments: a real-variable version is needed, and the constant degrades from 5/25/25/2 to (3+5)/2(3+\sqrt5)/2(3+5​)/2. A second obstacle is bookkeeping: the deviating player's load changes only on their own deviation, while the other players' randomization stays independent, and the expected cost of a player has to be related to expected facility loads, which uses linearity of the latencies in an essential way. The naive idea of applying Theorem 1 to each pure profile in the support of the equilibrium fails, because those profiles are not themselves Nash equilibria.

Formalization scope

Players and facilities are finite types ι and E; profiles are functions ι → Finset E, with feasibility IsProfile a separate predicate. Latencies are real-valued on natural-number loads, and "linear" means affine with nonnegative coefficients, fe(k)=aek+bef_e(k) = a_e k + b_efe​(k)=ae​k+be​; the paper displays only the identity latency fe(k)=kf_e(k)=kfe​(k)=k and states that its proofs extend. Mixed strategies are real weight functions on the finite strategy type Σi\Sigma_iΣi​; independence is built into AGT.profileProb; Nash deviations range over all lotteries, which is equivalent to pure deviations. agt_games maximizes payoffs, so the game is passed to it with payoff −ci-c_i−ci​. The goal is stated against every feasible pure profile rather than as a ratio, so no division by an optimum that may be 000 occurs. The social cost is the expected sum of the players' costs; the paper's second option, ∑eE[ne2]\sum_e \mathbb E[n_e^2]∑e​E[ne2​], is not part of this mission. A version with correlated distributions on profiles, or one that bounds the cost of an "averaged" profile instead of the expected cost, is a different statement and does not discharge the goal.

A complete development needs elementary finite-sum manipulation of product distributions (marginals of AGT.profileProb, expectations of loads), and a Jensen-type inequality (E[ne])2≤E[ne2](\mathbb E[n_e])^2 \le \mathbb E[n_e^2](E[ne​])2≤E[ne2​] for finite distributions. The expectation lemmas for congestion games are reusable for the other mixed results of the literature (weighted games, polynomial latencies). Proofs of the milestones, of the expectation layer, and alternative arguments are all welcome.

Selected references

  • G. Christodoulou, E. Koutsoupias, The Price of Anarchy of Finite Congestion Games, STOC 2005, pp. 67–73. https://doi.org/10.1145/1060590.1060600
  • B. Awerbuch, Y. Azar, A. Epstein, The Price of Routing Unsplittable Flow, STOC 2005, pp. 57–66. https://doi.org/10.1145/1060590.1060599
  • E. Koutsoupias, C. Papadimitriou, Worst-case Equilibria, STACS 1999, LNCS 1563, pp. 404–413. https://doi.org/10.1007/3-540-49116-3_38
  • R. W. Rosenthal, A Class of Games Possessing Pure-Strategy Nash Equilibria, International Journal of Game Theory 2 (1973), pp. 65–67. https://doi.org/10.1007/BF01737559
  • T. Roughgarden, É. Tardos, How Bad Is Selfish Routing?, Journal of the ACM 49 (2002), pp. 236–259. https://doi.org/10.1145/506147.506153
7 thms4 active usersReviewed
🏆Completed
Algorithmic Game TheoryConvex OptimizationOperations Research·Captain: mikedeng1

Consensus of Subjective Probabilities: The Pari-Mutuel Method: Equilibrium Track Probabilities Exist and Are UniqueResearch Paper

Motivation

A group of mmm individuals each hold a subjective probability distribution over the same nnn outcomes, and one wants a single distribution representing their consensus. Averaging and convolution are the obvious candidates. Eisenberg and Gale (Ann. Math. Statist. 30(1), 1959) observe that a real institution already performs such an aggregation: the pari-mutuel method of betting on horse races, in which the final "track's odds" on a horse are proportional to the total amount bet on it.

The difficulty is circular. Each bettor wants to bet where the ratio of their own probability to the track probability is largest, but the track probabilities are only known after everyone has bet. The paper asks whether track probabilities and bets compatible with both the bettors' strategies and the pari-mutuel principle exist, and whether they are determined by the data. It answers yes on both counts for the probabilities, which gives a well-defined notion of pari-mutuel consensus.

The variational problem the paper introduces, maximizing ∑ibilog⁡(utilityi)\sum_i b_i \log(\text{utility}_i)∑i​bi​log(utilityi​), is now known as the Eisenberg–Gale convex program. It is the standard tool for computing equilibria of linear Fisher markets, and the pari-mutuel market is the special case in which every bettor's utility for a horse is their subjective win probability.

Setting

There are mmm bettors B1,…,BmB_1,\dots,B_mB1​,…,Bm​ and nnn horses H1,…,HnH_1,\dots,H_nH1​,…,Hn​.

  • The subjective probability matrix P=(pij)P=(p_{ij})P=(pij​) is m×nm\times nm×n; pijp_{ij}pij​ is the probability, in the opinion of BiB_iBi​, that HjH_jHj​ wins. Each row is a probability distribution: pij≥0p_{ij}\ge 0pij​≥0 and ∑jpij=1\sum_j p_{ij}=1∑j​pij​=1.
  • Bettor BiB_iBi​ has a budget bi>0b_i>0bi​>0, with the unit of money chosen so that ∑ibi=1\sum_i b_i=1∑i​bi​=1.
  • Each column of PPP contains at least one positive entry (a horse nobody believes in can be removed).

Unknowns are Greek. πj\pi_jπj​ is the track probability of HjH_jHj​ and βij\beta_{ij}βij​ is the amount BiB_iBi​ bets on HjH_jHj​. Nonnegative πj,βij\pi_j,\beta_{ij}πj​,βij​ are equilibrium probabilities and bets when

(1) ∑j=1nβij=bi,(2) ∑i=1mβij=πj,(3) if μi=max⁡spisπs and βij>0, then μi=pijπj.\text{(1)}\ \sum_{j=1}^n\beta_{ij}=b_i,\qquad \text{(2)}\ \sum_{i=1}^m\beta_{ij}=\pi_j,\qquad \text{(3)}\ \text{if } \mu_i=\max_s\frac{p_{is}}{\pi_s}\text{ and }\beta_{ij}>0,\text{ then }\mu_i=\frac{p_{ij}}{\pi_j}.(1) j=1∑n​βij​=bi​,(2) i=1∑m​βij​=πj​,(3) if μi​=smax​πs​pis​​ and βij​>0, then μi​=πj​pij​​.

(1) is the budget relation, (2) the pari-mutuel condition, and (3) says each bettor bets only on horses that maximize the subjective expectation pij/πjp_{ij}/\pi_jpij​/πj​.

The paper's variational problem is

φ(ξ)=∑i=1mbilog⁡∑j=1npijξijonD={ξ: ξij≥0, ∑i=1mξij=1 for all j},\varphi(\xi)=\sum_{i=1}^m b_i\log\sum_{j=1}^n p_{ij}\xi_{ij}\quad\text{on}\quad D=\Big\{\xi:\ \xi_{ij}\ge0,\ \sum_{i=1}^m\xi_{ij}=1\ \text{for all } j\Big\},φ(ξ)=i=1∑m​bi​logj=1∑n​pij​ξij​onD={ξ: ξij​≥0, i=1∑m​ξij​=1 for all j},

with φ=−∞\varphi=-\inftyφ=−∞ where an inner sum vanishes. From a maximizer ξˉ\bar\xiξˉ​ it builds πj=max⁡ibipij/∑spisξˉis\pi_j=\max_i b_ip_{ij}/\sum_s p_{is}\bar\xi_{is}πj​=maxi​bi​pij​/∑s​pis​ξˉ​is​ (6) and βij=ξˉijπj\beta_{ij}=\bar\xi_{ij}\pi_jβij​=ξˉ​ij​πj​ (7). In Lean the market is PariMutuel.Consensus.Market m n, equilibrium is Market.IsEquilibrium, and φ\varphiφ, DDD, the maximizer predicate, (6) and (7) are Market.phi, D m n, Market.IsPhiMaximizer, Market.trackProb, Market.bets.

Formalization targets

Goal: existence and uniqueness of equilibrium probabilities

∃! π∈Rn  ∃ β∈Rm×n: (π,β) are equilibrium probabilities and bets.\exists!\,\pi\in\mathbb R^n\ \ \exists\,\beta\in\mathbb R^{m\times n}:\ (\pi,\beta)\ \text{are equilibrium probabilities and bets}.∃!π∈Rn  ∃β∈Rm×n: (π,β) are equilibrium probabilities and bets.

Only π\piπ is unique; the paper notes that equilibrium bets need not be.

Milestones

  1. φ\varphiφ attains its maximum on DDD at a point with every inner sum positive (p. 167).
  2. ∂φ/∂ξij=bipij/∑spisξis\partial\varphi/\partial\xi_{ij}=b_ip_{ij}/\sum_s p_{is}\xi_{is}∂φ/∂ξij​=bi​pij​/∑s​pis​ξis​ wherever the inner sums are positive (p. 167).
  3. (8): at a maximizer, ξˉij>0\bar\xi_{ij}>0ξˉ​ij​>0 implies πj=∂φ/∂ξˉij\pi_j=\partial\varphi/\partial\bar\xi_{ij}πj​=∂φ/∂ξˉ​ij​ (p. 167).
  4. Every πj\pi_jπj​ of (6) is positive (p. 167).
  5. EXISTENCE THEOREM: for every maximizer ξˉ\bar\xiξˉ​, (6)–(7) are equilibrium probabilities and bets (p. 167).
  6. Every equilibrium has πj>0\pi_j>0πj​>0 (p. 168).
  7. For two equilibria, ∑kπˉkπˉk/πk≤1\sum_k\bar\pi_k\bar\pi_k/\pi_k\le 1∑k​πˉk​πˉk​/πk​≤1 (p. 168).
  8. If π>0\pi>0π>0, πˉ≥0\bar\pi\ge0πˉ≥0, both sum to 1 and ∑kπˉk2/πk≤1\sum_k\bar\pi_k^2/\pi_k\le1∑k​πˉk2​/πk​≤1, then πˉ=π\bar\pi=\piπˉ=π (p. 168).
  9. UNIQUENESS THEOREM: equilibrium probabilities are unique (p. 168).

A further, non-milestone item states the referee's example (p. 168): with two bettors of equal budgets and two horses, if the first bettor's distribution is (12,12)(\tfrac12,\tfrac12)(21​,21​), the equilibrium probabilities are (12,12)(\tfrac12,\tfrac12)(21​,21​) whatever the second bettor believes.

Significance

The result makes pari-mutuel odds a well-defined function of the bettors' beliefs and budgets, so the consensus can be studied as a mathematical object; the referee's example shows it behaves very differently from averaging, since a single indifferent bettor can fix it. The existence proof replaces a fixed-point argument by a concave maximization, which is the origin of the Eisenberg–Gale program, later the basis of convex-programming and combinatorial algorithms for Fisher market equilibria.

The theorems are classical and fully proved in the paper. To the knowledge of this mission they have no machine-checked proof. The mission produces a formal pari-mutuel market model, the Eisenberg–Gale program with the correct treatment of log⁡0=−∞\log 0=-\inftylog0=−∞, and Lean proofs of existence and uniqueness. The platform's Market Equilibrium under Separable, Piecewise-Linear, Concave Utilities missions (Vazirani–Yannakakis) concern a related Fisher-market model with rational piecewise-linear utilities.

Difficulty

The existence statement, as the paper proves it, has two delicate points. φ\varphiφ is −∞-\infty−∞ on part of the boundary of DDD, so "continuous on a compact set" needs the extended-real reading, and a real-valued formalization must handle the boundary separately. The first-order condition (8) has to be derived from maximality on a polytope with equality constraints on columns, not from an unconstrained critical point.

For uniqueness, the natural first idea, strict concavity of φ\varphiφ, fails: φ\varphiφ is concave but not strictly concave in ξ\xiξ, and indeed equilibrium bets are not unique. Uniqueness has to be proved for the probabilities directly, for arbitrary equilibria and not only those built from a maximizer, and it needs positivity of all πj\pi_jπj​, which the paper uses without proof.

Formalization scope

  • Bettors are Fin m and horses Fin n, indexed from 0. All data are real. Every standing assumption (rows of PPP are probability vectors, no zero column, bi>0b_i>0bi​>0, ∑ibi=1\sum_i b_i=1∑i​bi​=1) is a field of Market; m≥1m\ge1m≥1 follows from ∑ibi=1\sum_ib_i=1∑i​bi​=1.
  • Condition (3) is written multiplied out: βij>0⇒pisπj≤pijπs\beta_{ij}>0\Rightarrow p_{is}\pi_j\le p_{ij}\pi_sβij​>0⇒pis​πj​≤pij​πs​ for all sss. This equals (3) when π>0\pi>0π>0 and encodes the paper's p/0=+∞p/0=+\inftyp/0=+∞ when some πs=0\pi_s=0πs​=0. Positivity of π\piπ is not part of the definition of equilibrium; it is milestone 6.
  • DDD has column sums one, as in (5).
  • φ\varphiφ is real-valued. A maximizer is a point of DDD with positive inner sums that dominates every point of DDD with positive inner sums; the excluded points have φ=−∞\varphi=-\inftyφ=−∞ on the page. The max in (6) is Finset.sup' over the nonempty set of bettors.
  • Ruled out: a version of (3) with real division (x/0=0x/0=0x/0=0) admits spurious equilibria with πs=0\pi_s=0πs​=0 and makes uniqueness false; a maximizer defined with the raw real φ\varphiφ (log⁡0=0\log0=0log0=0) changes the set of maximizers; adding π>0\pi>0π>0 or ∑jπj=1\sum_j\pi_j=1∑j​πj​=1 to the equilibrium definition weakens the goal.
  • Needed infrastructure: compactness of DDD and an argument handling the −∞-\infty−∞ boundary, one-variable derivatives of log⁡\loglog of linear forms, and the equality case of the Cauchy–Schwarz inequality. Milestones 7–9 use no analysis and can be attacked independently of 1–5. Proofs of any milestone, and reusable lemmas on the Eisenberg–Gale program, are welcome.

Selected references

  • E. Eisenberg and D. Gale, Consensus of subjective probabilities: the pari-mutuel method, The Annals of Mathematical Statistics 30(1):165–168, 1959. https://doi.org/10.1214/aoms/1177706369
  • V. V. Vazirani and M. Yannakakis, Market equilibrium under separable, piecewise-linear, concave utilities, Journal of the ACM 58(3), 2011. https://doi.org/10.1145/1970392.1970394
12 thms3 active usersReviewed
🏆Completed
Dynamic ProgrammingMarkov ChainOperations Research·Captain: mikedeng1

Discrete Dynamic Programming 2: A Stationary Policy Is Nearly Optimal as the Discount Factor Tends to 1 Exactly When It Maximizes x(g) and, Among Those, y(g)Research Paper

Motivation

A finite Markov decision problem with discounting is solved by Howard's policy improvement routine: start from a stationary policy, switch to actions that do better against its value, repeat. When the discount factor β\betaβ tends to 111 the total discounted income typically diverges, and the natural targets become the long-run average income and, among policies with the best average, the policy that does best in the transient phase. Howard treated this undiscounted case directly (Howard, 1960). David Blackwell's 1962 paper (Blackwell, 1962) treats β=1\beta = 1β=1 as a limit of β<1\beta < 1β<1: it expands the discounted return of a stationary policy in powers of 1−β1-\beta1−β and reads off which policies remain good as β→1\beta \to 1β→1. The two leading coefficients of that expansion, the gain x(f)x(f)x(f) and the bias y(f)y(f)y(f), became the standard objects of average-reward and sensitive-discount optimality (Veinott, 1969; Puterman, 1994, Ch. 8–10).

Timeline. Howard (1960) gives policy iteration for discounted and average-income problems. Blackwell (1962) proves that some stationary policy is optimal for all β\betaβ near 111 (his Theorem 5, the subject of a companion mission) and, in Theorem 4, characterizes the nearly optimal stationary policies through xxx and yyy. Miller and Veinott (1969) and Veinott (1969) extend the expansion to all orders (nnn-discount optimality).

Setting

There are finitely many states s∈Ss \in Ss∈S and a finite nonempty set AAA of actions. Action aaa in state sss pays an income i(s,a)∈Ri(s,a) \in \mathbb Ri(s,a)∈R and moves the system to s′s's′ with probability q(s′∣s,a)q(s' \mid s,a)q(s′∣s,a). FFF is the finite set of decision rules f:S→Af : S \to Af:S→A. A policy is a sequence π={f1,f2,… }\pi = \{f_1, f_2, \dots\}π={f1​,f2​,…} of decision rules; f(∞)f^{(\infty)}f(∞) uses fff every day, and (g,π)(g, \pi)(g,π) uses ggg first and then π\piπ. For f∈Ff \in Ff∈F, r(f)r(f)r(f) is the vector (i(s,f(s)))s(i(s,f(s)))_s(i(s,f(s)))s​ and Q(f)Q(f)Q(f) the Markov matrix (q(s′∣s,f(s)))s,s′(q(s' \mid s,f(s)))_{s,s'}(q(s′∣s,f(s)))s,s′​. The discounted return of π\piπ is the vector

Vβ(π)=∑n=0∞βnQ(f1)⋯Q(fn) r(fn+1),0≤β<1,V_\beta(\pi) = \sum_{n=0}^\infty \beta^n Q(f_1)\cdots Q(f_n)\, r(f_{n+1}), \qquad 0 \le \beta < 1,Vβ​(π)=n=0∑∞​βnQ(f1​)⋯Q(fn​)r(fn+1​),0≤β<1,

and Vβ(f)V_\beta(f)Vβ​(f) abbreviates Vβ(f(∞))V_\beta(f^{(\infty)})Vβ​(f(∞)). Vectors are compared coordinatewise; w1>w2w_1 > w_2w1​>w2​ means w1≥w2w_1 \ge w_2w1​≥w2​ and w1≠w2w_1 \neq w_2w1​=w2​. A policy is β-optimal if its return dominates that of every policy, and U(β)U(\beta)U(β) is the return of a β-optimal policy. It is optimal if it is β-optimal for all β\betaβ sufficiently near 111, and nearly optimal if U(β)−Vβ(π)→0U(\beta) - V_\beta(\pi) \to 0U(β)−Vβ​(π)→0 as β→1\beta \to 1β→1.

For any Markov matrix QQQ, the limit matrix Q∗Q^*Q∗ is the limit of (I+Q+⋯+QN)/(N+1)(I + Q + \cdots + Q^N)/(N+1)(I+Q+⋯+QN)/(N+1), and the deviation matrix is H=(I−Q+Q∗)−1−Q∗H = (I - Q + Q^*)^{-1} - Q^*H=(I−Q+Q∗)−1−Q∗. For a rule fff, Q∗(f)Q^*(f)Q∗(f) and H(f)H(f)H(f) are those of Q(f)Q(f)Q(f), and

x(f)=Q∗(f) r(f),y(f)=H(f) r(f).x(f) = Q^*(f)\, r(f), \qquad y(f) = H(f)\, r(f).x(f)=Q∗(f)r(f),y(f)=H(f)r(f).

With p(s,a)w=∑s′q(s′∣s,a)ws′p(s,a)w = \sum_{s'} q(s' \mid s,a) w_{s'}p(s,a)w=∑s′​q(s′∣s,a)ws′​, the set G(s,f)G(s,f)G(s,f) consists of the actions aaa with p(s,a)x(f)>xs(f)p(s,a)x(f) > x_s(f)p(s,a)x(f)>xs​(f), or with p(s,a)x(f)=xs(f)p(s,a)x(f) = x_s(f)p(s,a)x(f)=xs​(f) and i(s,a)+p(s,a)y(f)>xs(f)+ys(f)i(s,a) + p(s,a)y(f) > x_s(f) + y_s(f)i(s,a)+p(s,a)y(f)>xs​(f)+ys​(f); E(s,f)E(s,f)E(s,f) consists of those with equality in both.

Formalization targets

Goal: Theorem 4(e)

For any f0f_0f0​ with G(s,f0)=∅G(s,f_0) = \varnothingG(s,f0​)=∅ for all sss:

x(f0)≥x(g)  ∀g∈F;∃f∗∈F∗:={g:x(g)=x(f0)} with y(f∗)≥y(g) ∀g∈F∗;x(f_0) \ge x(g)\ \ \forall g \in F;\qquad \exists f^* \in F^* := \{g : x(g) = x(f_0)\}\ \text{with}\ y(f^*) \ge y(g)\ \forall g \in F^*;x(f0​)≥x(g)  ∀g∈F;∃f∗∈F∗:={g:x(g)=x(f0​)} with y(f∗)≥y(g) ∀g∈F∗; g(∞) is nearly optimal  ⟺  x(g)=x(f∗) and y(g)=y(f∗).g^{(\infty)} \text{ is nearly optimal} \iff x(g) = x(f^*) \text{ and } y(g) = y(f^*).g(∞) is nearly optimal⟺x(g)=x(f∗) and y(g)=y(f∗).

Milestones and intermediate results

Milestones: Lemma 1(b) (rank⁡(I−Q)+rank⁡Q∗=S\operatorname{rank}(I-Q) + \operatorname{rank} Q^* = Srank(I−Q)+rankQ∗=S), Theorem 4(b) (improvement for β near 1), 4(c) (a sufficient condition for optimality), Lemma 2, and 4(d) (a sufficient condition for near optimality).

The mission also states, as intermediate results:

  • Lemma 1(a), (c), (d): for every Markov matrix, convergence of the Cesàro means to a Markov Q∗Q^*Q∗ with QQ∗=Q∗Q=Q∗Q∗=Q∗QQ^* = Q^*Q = Q^*Q^* = Q^*QQ∗=Q∗Q=Q∗Q∗=Q∗; unique solvability of Qx=xQx = xQx=x, Q∗x=Q∗cQ^*x = Q^*cQ∗x=Q∗c; nonsingularity of I−Q+Q∗I - Q + Q^*I−Q+Q∗, ∑nβn(Qn−Q∗)→H\sum_n \beta^n (Q^n - Q^*) \to H∑n​βn(Qn−Q∗)→H and the identities for HHH.
  • Theorem 4(a): Vβ(f)=x(f)/(1−β)+y(f)+o(1)V_\beta(f) = x(f)/(1-\beta) + y(f) + o(1)Vβ​(f)=x(f)/(1−β)+y(f)+o(1), with x(f),y(f)x(f), y(f)x(f),y(f) the unique solutions of their linear systems; display (2), the same expansion for (g,f(∞))(g, f^{(\infty)})(g,f(∞)).
  • Theorem 3 and its Corollary for fixed β<1\beta < 1β<1, and the first assertion of 4(e).

Significance

Theorem 4(e) says that near optimality for β near 1 is exactly lexicographic maximization: first of the average income xxx, then of the bias yyy. It justifies the two-level optimality equations used throughout average-reward dynamic programming and shows that, once the β = 1 improvement routine stops, the remaining problem is a bias maximization over the gain-optimal rules. Theorem 4(a) is the first two terms of the Laurent expansion of discounted values, the starting point of sensitive-discount optimality.

The results are classical and proved in the paper (Lemma 1 with a reference to Kemeny and Snell); no machine-checked proof of them is known on the platform. A complete development produces a multichain theory of Cesàro limit and deviation matrices of arbitrary finite Markov matrices, which Mathlib does not have, and the expansion of discounted returns near β = 1.

Difficulty

Lemma 1 must be proved for every Markov matrix, including reducible and periodic ones, where QnQ^nQn does not converge and the stationary distribution is not unique; arguments through the Perron–Frobenius eigenvector of an irreducible chain do not apply. In Theorem 4(e) the hard part is the existence of a single f∗f^*f∗ whose bias dominates every gain-optimal rule in every coordinate at once; a rule maximizing each coordinate separately is not enough. The final characterization compares a stationary policy with all policies, including time-dependent ones, through U(β)U(\beta)U(β).

Formalization scope

States and actions are finite nonempty types; incomes are real of any sign; a policy is a sequence ℕ → (St → Act) with π 0 the paper's f1f_1f1​. VβV_\betaVβ​ is a real tsum. Q∗Q^*Q∗ is limUnder of the Cesàro means, and its existence is Lemma 1(a), not an assumption; H(β)H(\beta)H(β) is a matrix tsum, whose summability for 0≤β<10 \le \beta < 10≤β<1 is part of Lemma 1(d); HHH uses Mathlib's total inverse, whose nonsingularity is also part of Lemma 1(d). x(f)x(f)x(f) and y(f)y(f)y(f) are defined by the closed forms Q∗(f)r(f)Q^*(f)r(f)Q∗(f)r(f) and H(f)r(f)H(f)r(f)H(f)r(f) from the paper's proof, and Theorem 4(a) asserts that they are the unique solutions of the paper's defining systems. Limits "as β → 1" are along β→1−\beta \to 1^-β→1−. "Nearly optimal" is encoded without UUU: for every ε>0\varepsilon > 0ε>0, for all β in some interval (β0,1)(\beta_0, 1)(β0​,1), every policy's return is at most Vβ(π)+εV_\beta(\pi) + \varepsilonVβ​(π)+ε in every coordinate; this is equivalent to U(β)−Vβ(π)→0U(\beta) - V_\beta(\pi) \to 0U(β)−Vβ​(π)→0 because a β-optimal policy exists. "Optimal" (§4) and "β-optimal" (§3) are distinct definitions, and Theorem 3's β-dependent improvement set is distinct from the §4 set G(s,f)G(s,f)G(s,f).

A formalization in which optimality or near optimality is tested only against stationary policies, or in which Q∗Q^*Q∗ is assumed to exist or the chain to be irreducible, proves a different and easier theorem and does not meet the targets.

Contributions are welcome at every level: the Cesàro and Abel limit theory of finite Markov matrices (reusable well beyond this paper), the policy improvement theorem for fixed β, and the comparison arguments of Theorem 4. Theorem 3 and the Corollary are also drafted in the companion mission on Theorem 5 in another namespace.

Selected references

  • D. Blackwell, Discrete Dynamic Programming, Ann. Math. Statist. 33(2):719–726, 1962. https://doi.org/10.1214/aoms/1177704593
  • R. A. Howard, Dynamic Programming and Markov Processes, MIT Press, 1960.
  • J. G. Kemeny and J. L. Snell, Finite Markov Chains, Van Nostrand, 1960.
  • B. L. Miller and A. F. Veinott, Discrete Dynamic Programming with a Small Interest Rate, Ann. Math. Statist. 40(2):366–370, 1969.
  • A. F. Veinott, Discrete Dynamic Programming with Sensitive Discount Optimality Criteria, Ann. Math. Statist. 40(5):1635–1660, 1969. https://doi.org/10.1214/aoms/1177697379
  • M. L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming, Wiley, 1994. https://doi.org/10.1002/9780470316887
10 thms2 active usersReviewed
🏆Completed
Algorithmic Game TheoryOperations ResearchProbability+1·Captain: mikedeng1

Secretary Problems: Weights and Discounts 1: An (8+3e)-Competitive Algorithm for the Weighted Secretary ProblemResearch Paper

Motivation

The classical secretary problem asks how to select one valuable candidate when candidates arrive in random order and a decision must be made when each candidate appears. Many allocation settings have several goods of unequal quality instead of a single position. An employer may have roles of different desirability, or a seller may have placements with different visibility. In the weighted secretary problem, an agent's value is multiplied by the weight of the good assigned to that agent. The algorithm must decide irrevocably as agents arrive, while the benchmark sees every value before assigning goods. Babaioff, Dinitz, Gupta, Immorlica and Talwar study this model with arbitrary fixed agent values and a uniformly random arrival order, and give a constant competitive ratio independent of the number of agents and goods (authors' version, §§2–3).

The paper also studies time discounts and matroid constraints. This mission concerns its weighted-goods result, Theorem 3.4. The result combines an online allocation rule for several comparably valuable agents with the familiar one-choice secretary rule for an unusually valuable agent. These are distinct ways in which the sorted offline assignment can earn value; both are present even when the weights are fixed in advance. The weighted model matters because matching a valuable agent to an unsuitable good can lose value despite accepting the right agent.

Setting

There are nnn agents e∈Ue\in Ue∈U, each with a nonnegative value v(e)v(e)v(e), and KKK goods indexed in decreasing order of nonnegative weight:

w(1)≥w(2)≥⋯≥w(K)≥0.w(1)\ge w(2)\ge\cdots\ge w(K)\ge0.w(1)≥w(2)≥⋯≥w(K)≥0.

An assignment sss gives each good to at most one agent, and each agent receives at most one good. A good may remain unassigned, represented by ⊥\bot⊥ with v(⊥)=0v(\bot)=0v(⊥)=0. Its value is ∑k=1Kv(s(k))w(k)\sum_{k=1}^K v(s(k))w(k)∑k=1K​v(s(k))w(k). Agent values are arbitrary, not drawn independently from a distribution. The uncertainty is the arrival order π\piπ, chosen uniformly from all permutations; an agent's value becomes visible on arrival, and an allocation decision cannot be revised.

The offline optimum, OPT\mathrm{OPT}OPT, assigns the heaviest good to the highest-valued agent, the next good to the next agent, and so on. If K>nK>nK>n, the extra goods remain unassigned. A consistent tie break makes the ordering unique without changing the numerical value. This sorted assignment is defined directly; the mission does not replace it with an unconstrained variable said to be optimal.

The reservation algorithm draws a sample size τ∼Binom(n,1/2)\tau\sim\mathrm{Binom}(n,1/2)τ∼Binom(n,1/2), observes the first τ\tauτ agents without allocation, and retains the best min⁡(K,τ)\min(K,\tau)min(K,τ) sampled agents. Positive values are grouped into value classes [2i−1,2i)[2^{i-1},2^i)[2i−1,2i) for integer iii. A sampled agent in class iii reserves one good in that class's contiguous block, with higher classes receiving heavier blocks. A later agent receives the heaviest unassigned good reserved for its class when one is available. The classical secretary rule instead observes the first ⌊n/e⌋\lfloor n/e\rfloor⌊n/e⌋ agents, then selects the first later arrival better than every predecessor; its winner receives good 111.

Formalization targets

The mission's goal is the exact guarantee of Theorem 3.4 for Algorithm AAA, which runs the reservation algorithm with probability 8/(3e+8)8/(3e+8)8/(3e+8) and the classical rule with probability 3e/(3e+8)3e/(3e+8)3e/(3e+8):

OPT≤(8+3e) E[A].\mathrm{OPT}\le(8+3e)\,\mathbb E[A].OPT≤(8+3e)E[A].

Here the expectation covers the uniform arrival permutation, the independent binomial sample size used by the reservation branch, and the mixing coin. The multiplicative inequality expresses competitiveness even when an expected payoff is zero. It uses the explicit constant in the paper's proof rather than an instance-dependent or unspecified constant.

Four source results form the milestones. The classical secretary rule selects the maximum with probability at least 1/e1/e1/e. Lemma 3.2 compares the starting indices bib_ibi​ and oio_ioi​ of class-iii blocks in the reservation and optimum assignments. Lemma 3.1 says that if the optimum assigns at least two agents from class iii, the reservation rule assigns at least ui/4u_i/4ui​/4 agents from that class in expectation. Lemma 3.3 converts this to expected value at least OPTi/8\mathrm{OPT}_i/8OPTi​/8. The target retains the paper's class condition and both numerical fractions (authors' version, pp. 4–5).

Significance

Theorem 3.4 supplies a constant factor guarantee for irrevocable allocation when goods have different weights and agents arrive in random order. The factor does not grow with nnn or KKK. It separates the effects of uncertain arrivals from the offline matching of high values to high weights, and it supplies a benchmark for later variants with more complicated feasibility constraints. The paper extends the reservation idea to additional combinatorial settings, including partition-matroid variants in Appendix C (authors' version, Appendix C).

The theorem is proved in the source paper, while the Lean statements in this mission are proof obligations. Formalizing them requires checking that the random-order model, sample distribution, tie convention and assignments jointly express the same algorithm. A complete development will also establish reusable finite-average facts for random permutations and binomial samples, and structural facts about sorted assignments and reserved blocks. Those pieces can support other secretary problems in the series; the mission's specific promise remains the weighted algorithm's exact bound.

Difficulty

A count of how many agents a class receives does not by itself control the weighted value of those goods. Goods have unequal weights, and the value of assigning the next good changes with its position in a block. A class whose offline optimum receives several agents can also lose all its sampled members from the allocation phase. Thus a direct comparison of expected class counts with expected class values is insufficient. The paper's separate count, block-position and value statements identify the claims a solver must establish; the final theorem must also account for classes represented only once in the offline assignment (authors' version, p. 5).

Formalization scope

Agents and goods are Fin n and Fin K; their indices start at zero in Lean, so paper time ttt corresponds to Lean index t−1t-1t−1. An arrival permutation maps time to agent. Values and weights are real and explicitly nonnegative, and weights are antitone in the good index. The finite sums defining expectations are normalized by n!n!n! for permutations and by (nτ)/2n\binom n\tau/2^n(τn​)/2n for sample sizes. No measurability or integration convention is needed. For the goal, K≥1K\ge1K≥1 makes the heaviest good available; K>nK>nK>n is allowed.

Equal values are ordered by smaller original agent index throughout the sorted optimum, the sample's top agents and the classical rule. The classical rule observes exactly ⌊n/e⌋\lfloor n/e\rfloor⌊n/e⌋ arrivals, and zero-valued agents reserve no value-class goods. Positive values below one use negative integer class indices. The paper says only that class iii holds the values “between” 2i−12^{i-1}2i−1 and 2i2^i2i (p. 4, and again in Appendix C, p. 12); the mission fixes the half-open interval [2i−1,2i)[2^{i-1},2^i)[2i−1,2i), so that the classes partition the positive reals (authors' version, pp. 4, 12). A reservation assignment is built from each post-sample agent's rank within its class, so a good is offered to at most one such agent. The theorem is about this concrete algorithm and the concrete sorted offline assignment; an arbitrary favorable policy or an optimum supplied as a hypothesis would not express the source result.

The development needs a finite assignment interface, a tie-aware rank order, value classes, the two online rules, and normalized finite expectations. The assignment and finite-average definitions are reusable. Contributions that prove the structural validity of the reservation assignment, the classical success guarantee, Lemmas 3.1–3.3, or the final combination all advance the stated target.

Selected references

  • Moshe Babaioff, Michael Dinitz, Anupam Gupta, Nicole Immorlica and Kunal Talwar, Secretary Problems: Weights and Discounts, Proceedings of SODA 2009; authors' full version, proceedings DOI.
7 thms2 active usersReviewed
🏆Completed
CombinatoricsGraph TheoryOperations Research+2·Captain: mikedeng1

Secretary Problems: Weights and Discounts 5: A 3e-Competitive Algorithm for the Graphic Matroid Secretary ProblemResearch Paper

Motivation

In the secretary problem, nnn items with nonnegative values arrive one at a time in a uniformly random order, and an online algorithm must decide on each arrival, irrevocably, whether to keep it. The classical version keeps one item; the rule that observes a 1/e1/e1/e fraction of the arrivals and then takes the first item better than everything seen picks the best item with probability at least 1/e1/e1/e (Ferguson 1989).

Babaioff, Immorlica and Kleinberg (SODA 2007; journal version J. ACM 2018) introduced the matroid secretary problem: the kept set must be independent in a known matroid. It models online auctions in which the feasible sets of winners have matroid structure, for example hiring along the edges of a network without closing a cycle. They gave a 161616-competitive algorithm when the matroid is graphic, i.e. the items are the edges of a graph and a set is feasible when it contains no cycle.

Timeline for graphic matroids:

  • 2007, Babaioff–Immorlica–Kleinberg: 161616-competitive.
  • 2009, Babaioff–Dinitz–Gupta–Immorlica–Talwar (SODA 2009, Theorem 1.5): 3e≈8.153e\approx 8.153e≈8.15-competitive, through a random reduction to partition matroids. This mission formalizes that result.
  • 2009, Korula–Pál (ICALP 2009): 2e2e2e-competitive, by a different reduction.

Setting

Let G=(V,E)G=(V,E)G=(V,E) be a finite simple graph. Each edge eee has a value v(e)≥0v(e)\ge 0v(e)≥0. A set S⊆ES\subseteq ES⊆E is independent in the graphic matroid of GGG if the graph (V,S)(V,S)(V,S) has no cycle. The offline optimum is

OPT(G,v)=max⁡{∑e∈Sv(e):S⊆E acyclic}.\mathrm{OPT}(G,v)=\max\Big\{\sum_{e\in S}v(e): S\subseteq E\ \text{acyclic}\Big\}.OPT(G,v)=max{e∈S∑​v(e):S⊆E acyclic}.

The edges arrive in a uniformly random order. An algorithm sees each edge and its value on arrival and decides at once whether to select it. The selected set must be acyclic. The algorithm is α\alphaα-competitive if OPT(G,v)≤α⋅E[value of the selected set]\mathrm{OPT}(G,v)\le\alpha\cdot\mathbb E[\text{value of the selected set}]OPT(G,v)≤α⋅E[value of the selected set] for every GGG and every v≥0v\ge 0v≥0.

A partition matroid on a subset U′⊆EU'\subseteq EU′⊆E is given by a family PPP of nonempty, pairwise disjoint parts with union U′U'U′: a set is independent when it lies in U′U'U′ and meets each part at most once. Its max-weight base has value val(P,v)=∑p∈Pmax⁡e∈pv(e)\mathrm{val}(P,v)=\sum_{p\in P}\max_{e\in p}v(e)val(P,v)=∑p∈P​maxe∈p​v(e).

Definition 5.1. A random partition μ\muμ (a probability distribution on such families, chosen from GGG alone) is an α\alphaα-partition scheme if every partition in its support has only acyclic independent sets, and for every v≥0v\ge 0v≥0,

OPT(G,v)≤α⋅EP∼μ[val(P,v)].\mathrm{OPT}(G,v)\le \alpha\cdot\mathbb E_{P\sim\mu}[\mathrm{val}(P,v)].OPT(G,v)≤α⋅EP∼μ​[val(P,v)].

The random partition of Lemma 5.3. Pick an edge {u,w}\{u,w\}{u,w} uniformly at random. With probability 12\tfrac1221​ colour uuu red and www blue, otherwise the reverse. Colour every other vertex red or blue independently with probability 12\tfrac1221​. Each red vertex xxx gets a part: the red-blue edges at xxx. Then repeat on the edges with both endpoints blue, with fresh randomness.

The algorithm. Draw the partition, let the edges arrive, and on each part run the classical secretary rule on that part's arrivals. Output all selected edges.

Formalization targets

Goal: Theorem 1.5

For every finite simple graph GGG and every v≥0v\ge 0v≥0:

  1. every possible output of the algorithm is an acyclic set of edges of GGG;
OPT(G,v)≤3e⋅E[ALG].\mathrm{OPT}(G,v)\le 3e\cdot\mathbb E[\mathrm{ALG}].OPT(G,v)≤3e⋅E[ALG].

Part 1 is needed for the statement to have content: an algorithm that selects every edge would otherwise satisfy part 2.

Milestones

  • Section 2, p. 4. On m≥1m\ge1m≥1 arrivals, the classical rule selects the maximum with probability at least 1/e1/e1/e.
  • Theorem 5.4, first clause. For a fixed partition PPP, the per-part rule outputs a set independent in the partition matroid, and val(P,v)≤e⋅Eπ[ALG]\mathrm{val}(P,v)\le e\cdot\mathbb E_\pi[\mathrm{ALG}]val(P,v)≤e⋅Eπ​[ALG].
  • Lemma 5.3, independence. Every partition the random construction can produce is a partition matroid on a subset of EEE, and each of its independent sets is a forest.
  • Lemma 5.3. The construction is a 333-partition scheme.
  • Section 5, p. 10. Any α\alphaα-partition scheme for a graphic matroid, combined with the per-part rule, gives a feasible, eαe\alphaeα-competitive algorithm.

Significance

The theorem shows that the graphic matroid secretary problem admits a constant-competitive algorithm with a small explicit constant. It does so through a reduction: a random partition matroid that is feasible for the original matroid and loses only a constant factor in expectation. The reduction separates the combinatorics (Lemma 5.3) from the online part (Theorem 5.4). The same framework gives algorithms for uniform and transversal matroids and for the weighted and discounted variants on any matroid with an α\alphaα-partition property.

The result is proved in the paper; it has not been formalized. The mission contributes a machine-checked version of the reduction, a formal treatment of a recursively defined random partition, and the classical secretary bound in a reusable finite form. The constant 3e3e3e is not the best known for graphic matroids (Korula–Pál improve it to 2e2e2e), so the formal goal is this algorithm's guarantee, not the best possible ratio.

Difficulty

The online half is routine once the classical bound is available: the relative order of the edges in each part is uniform, and the parts are disjoint. The difficulty is Lemma 5.3. The natural idea of using a fixed optimal forest to build the partition is ruled out because the partition must be chosen before the values are seen. The expectation bound must therefore hold for every valuation at once, for a law that depends on the graph only. The construction is recursive and random: its expected value is not a closed-form sum, and any bound has to be carried through the random sequence of blue-blue subgraphs. Feasibility needs an invariant across rounds: the parts created later live inside the blue-blue edges of every earlier round.

Formalization scope

  • Graph. A SimpleGraph on a Fintype vertex type with decidable adjacency. The edges are G.edgeFinset, and acyclicity of SSS is (SimpleGraph.fromEdgeSet S).IsAcyclic. Multigraphs are not covered.
  • Values. Values are a real function v : Sym2 V → ℝ with ∀ e, 0 ≤ v e; only the values on edges matter.
  • OPT is a Finset.sup' over acyclic subsets of the edge set. A partition is a finite family of nonempty, pairwise disjoint parts inside the edge set. Its max-weight base value is the sum of the part maxima.
  • Random partition. A PMF defined by well-founded recursion on the number of edges. Empty parts are dropped, and edges with two red endpoints are discarded.
  • Random order. The edges are numbered by a fixed enumeration. An arrival order is a permutation of the numbers, and expectation over the order is the average over all ∣E∣!|E|!∣E∣! permutations.
  • Classical rule. It samples ⌊m/e⌋\lfloor m/e\rfloor⌊m/e⌋ arrivals of a part with mmm edges. Ties are broken by preferring the smaller edge number among equal values.
  • Constants. Competitiveness is multiplicative (OPT≤3e⋅E[ALG]\mathrm{OPT}\le 3e\cdot\mathbb E[\mathrm{ALG}]OPT≤3e⋅E[ALG]), so a zero expectation is not a loophole.
  • Ruling out trivial formalizations. In Definition 5.1 the random partition is fixed before the valuation, and the independence requirement holds for every partition in its support. A partition allowed to depend on vvv would make every matroid 111-partitionable.

A complete development needs the classical secretary bound in finite form, the uniformity of induced sub-orders of a uniform permutation, expectations of PMF.bind along a well-founded recursion, and facts about forests in SimpleGraph. The first two, and a general graphic-matroid layer, are reusable beyond this mission. Proofs of any milestone, alternative proofs of Lemma 5.3, and extensions to the uniform and transversal cases of Theorem 5.2 are welcome.

Selected references

  • M. Babaioff, M. Dinitz, A. Gupta, N. Immorlica, K. Talwar, Secretary Problems: Weights and Discounts, Proc. 20th ACM-SIAM Symposium on Discrete Algorithms (SODA), 2009. https://doi.org/10.1137/1.9781611973068.135
  • M. Babaioff, N. Immorlica, R. Kleinberg, Matroids, secretary problems, and online mechanisms, SODA 2007, pp. 434–443. https://dl.acm.org/doi/10.5555/1283383.1283429
  • M. Babaioff, N. Immorlica, D. Kempe, R. Kleinberg, Matroid Secretary Problems, Journal of the ACM 65(6), 2018. https://doi.org/10.1145/3212512
  • N. Korula, M. Pál, Algorithms for Secretary Problems on Graphs and Hypergraphs, ICALP 2009, LNCS 5556. https://doi.org/10.1007/978-3-642-02930-1_42
  • T. S. Ferguson, Who solved the secretary problem?, Statistical Science 4(3), 1989. https://doi.org/10.1214/ss/1177012493
10 thms3 active usersReviewed
Operations ResearchProbabilityStatistics·Captain: mikedeng1

Conditional Logit Analysis of Qualitative Choice Behavior 5: The Maximum Likelihood Estimator Exists with Probability Tending to One and Is Consistent and Asymptotically NormalResearch Paper

Motivation

The conditional logit model is the workhorse of discrete choice analysis in econometrics, transportation planning, marketing and revenue management. An individual facing a finite set of alternatives picks alternative iii with probability proportional to eziθe^{z_i\theta}ezi​θ, where ziz_izi​ is a vector of observed attributes and θ\thetaθ an unknown parameter vector. Daniel McFadden's 1974 chapter, Conditional Logit Analysis of Qualitative Choice Behavior, derived this model from a theory of random utility maximization and set out how to estimate θ\thetaθ by maximum likelihood. McFadden received the 2000 Nobel Memorial Prize in Economic Sciences for his development of theory and methods for analyzing discrete choice.

Every confidence interval and hypothesis test computed from a fitted logit model rests on the large-sample theory in §III of that chapter: the maximum likelihood estimator exists with probability tending to one, converges to the true parameter, and is approximately normal with covariance given by the inverse information matrix. This mission formalizes that theory, Lemmas 5 and 6 of the paper, as proved in its Appendix.

Setting

Observations are indexed serially, m=0,1,2,…m = 0, 1, 2, \dotsm=0,1,2,…, as in the paper's Appendix ("Let m be a serial index of trials and repetitions"). Observation mmm offers Jm≥1J_m \ge 1Jm​≥1 alternatives, and alternative iii carries a vector zim∈RKz_{im} \in \mathbb R^Kzim​∈RK of independent variables. For a parameter θ∈RK\theta \in \mathbb R^Kθ∈RK the selection probabilities are

Pim(θ)=ezimθ∑j=1Jmezjmθ,zˉm(θ)=∑iPim(θ) zim.P_{im}(\theta) = \frac{e^{z_{im}\theta}}{\sum_{j=1}^{J_m} e^{z_{jm}\theta}}, \qquad \bar z_m(\theta) = \sum_{i} P_{im}(\theta)\, z_{im}.Pim​(θ)=∑j=1Jm​​ezjm​θezim​θ​,zˉm​(θ)=i∑​Pim​(θ)zim​.

The data are generated at a true parameter θ0\theta^0θ0: the chosen alternatives Y0,Y1,…Y_0, Y_1, \dotsY0​,Y1​,… are independent random variables with Pr⁡(Ym=i)=Pim(θ0)\Pr(Y_m = i) = P_{im}(\theta^0)Pr(Ym​=i)=Pim​(θ0). The log-likelihood of the first qqq observations is Lq(θ)=∑m<qlog⁡PYmm(θ)L^q(\theta) = \sum_{m<q}\log P_{Y_m m}(\theta)Lq(θ)=∑m<q​logPYm​m​(θ). The moment matrix of observation mmm is

Ωm=∑iPim(θ0) (zim−zˉm)(zim−zˉm)′,zˉm=zˉm(θ0).\Omega_m = \sum_{i} P_{im}(\theta^0)\,(z_{im}-\bar z_m)(z_{im}-\bar z_m)', \qquad \bar z_m = \bar z_m(\theta^0).Ωm​=i∑​Pim​(θ0)(zim​−zˉm​)(zim​−zˉm​)′,zˉm​=zˉm​(θ0).

Axiom 7 asks that Jm≤J∗J_m \le J_*Jm​≤J∗​ and ∣zim∣≤M|z_{im}| \le M∣zim​∣≤M uniformly, and that 1q∑m<qΩm\frac1q\sum_{m<q}\Omega_mq1​∑m<q​Ωm​ converge to a positive definite matrix Ω\OmegaΩ. Axiom 6, for a given sample, asks that no nonzero γ\gammaγ satisfy (zjm−zYmm)γ≤0(z_{jm} - z_{Y_m m})\gamma \le 0(zjm​−zYm​m​)γ≤0 for all observed mmm and all jjj. A maximum likelihood estimator θ^q\hat\theta^qθ^q is a measurable choice of a maximizer of LqL^qLq, wherever one exists.

Formalization targets

Goal: Lemma 6

θ^q→Pr⁡θ0andq Ω1/2(θ^q−θ0)→dN(0,IK)(q→∞).\hat\theta^q \xrightarrow{\Pr} \theta^0 \quad\text{and}\quad \sqrt q\,\Omega^{1/2}(\hat\theta^q - \theta^0) \xrightarrow{d} N(0, I_K) \qquad (q \to \infty).θ^qPr​θ0andq​Ω1/2(θ^q−θ0)d​N(0,IK​)(q→∞).

Milestones

  1. Axiom 7 implies Axiom 5 (the full-rank condition) in all sufficiently large samples.
  2. Equation (42): Pim(θ)≥1/(J∗e2M∣θ∣)P_{im}(\theta) \ge 1/(J_* e^{2M|\theta|})Pim​(θ)≥1/(J∗​e2M∣θ∣).
  3. Lemma 5: Pr⁡(Axiom 6 holds and Lq attains its maximum)→1\Pr(\text{Axiom 6 holds and } L^q \text{ attains its maximum}) \to 1Pr(Axiom 6 holds and Lq attains its maximum)→1.
  4. Equation (43): the first three derivatives of log⁡Pim\log P_{im}logPim​ are bounded by 2M2M2M, 4M24M^24M2, 8M38M^38M3.
  5. Equation (46): each score ∇log⁡PYmm(θ0)\nabla\log P_{Y_m m}(\theta^0)∇logPYm​m​(θ0) has mean zero.
  6. Equation (47): each expected Hessian equals −Ωm-\Omega_m−Ωm​.
  7. Consistency of θ^q\hat\theta^qθ^q.
  8. Equation (58): q−1/2 Ω−1/2∑m<q∇log⁡PYmm(θ0)→dN(0,IK)q^{-1/2}\,\Omega^{-1/2}\sum_{m<q}\nabla\log P_{Y_m m}(\theta^0) \xrightarrow{d} N(0, I_K)q−1/2Ω−1/2∑m<q​∇logPYm​m​(θ0)d​N(0,IK​).

Significance

The result. Lemma 6 is what licenses reading θ^q\hat\theta^qθ^q as approximately N(θ0,q−1Ω−1)N(\theta^0, q^{-1}\Omega^{-1})N(θ0,q−1Ω−1), so that the diagonal of the inverse information matrix estimates the sampling variances and q(θ^q−θ0)′Ω(θ^q−θ0)q(\hat\theta^q-\theta^0)'\Omega(\hat\theta^q-\theta^0)q(θ^q−θ0)′Ω(θ^q−θ0) is asymptotically χK2\chi^2_KχK2​. Lemma 5 complements it: in finite samples the likelihood can fail to have a maximum (the observations are then "explained" by a direction γ\gammaγ of Axiom 6), and the lemma shows this failure is asymptotically negligible. The data are not identically distributed (each observation has its own alternatives), so the result is not an instance of the textbook i.i.d. maximum likelihood theorem.

Formalizing it. The results are proved in the paper, in outline. A machine-checked version adds: a complete proof of the existence part (Lemma 5), whose published argument is a sketch by induction over an infinite index set; a precise treatment of the estimator where no maximizer exists; the correction of two misprints in the published proof (the normalization 1/q1/q1/q in (58), which must be 1/q1/\sqrt q1/q​, and a constant in (51)); and a multivariate Lindeberg–Feller central limit theorem for bounded, independent, non-identically distributed vectors, which the proof invokes and which is reusable well beyond this paper. No machine-checked proof of these results is known.

Difficulty

The obvious route, "the log-likelihood is concave, so its maximizer converges", needs a maximizer to exist, and in a finite sample it may not; the estimator is defined only on an event whose probability must first be shown to tend to one. Consistency then needs a uniform law of large numbers for the gradient on a sphere around θ0\theta^0θ0, controlled by the third-derivative bound (43). Asymptotic normality needs a central limit theorem for independent but not identically distributed score vectors, with covariances Ωm\Omega_mΩm​ that converge only on average; the i.i.d. central limit theorem does not apply. Finally the random Hessian at an intermediate point must be shown to converge in probability, which ties the consistency result into the normality argument.

Formalization scope

  • Vectors live in EuclideanSpace ℝ (Fin K); zθz\thetazθ is the inner product, and all norms are Euclidean (footnote 11's sum-of-absolute-values norm is equivalent and gives the same qualitative axiom); derivative bounds use operator norms.
  • The paper's NNN trials with RnR_nRn​ repetitions are the special case of the serial indexing in which consecutive observations repeat their data; the sample size ∑nRn\sum_n R_n∑n​Rn​ is qqq.
  • Axiom 7's limit (27) is taken in its serial form (48), with PPP evaluated at θ0\theta^0θ0.
  • The estimator is any measurable selection that maximizes LqL^qLq whenever LqL^qLq has a maximum, and is unconstrained otherwise. Requiring a maximizer for every sample would be unsatisfiable, since Axiom 6 fails with positive probability, and would make the goal vacuous; this convention rules that out.
  • Consistency is TendstoInMeasure. Asymptotic normality is TendstoInDistribution to a random vector whose law is stdGaussian. Ω1/2\Omega^{1/2}Ω1/2 is the positive semidefinite square root CFC.sqrt.
  • Needed infrastructure: derivatives of log-sum-exp, a law of large numbers for bounded independent vectors, and a multivariate Lindeberg–Feller theorem. Mathlib provides the one-dimensional i.i.d. central limit theorem only. Contributions of these general results as separate theorems are welcome.

Selected references

  • D. McFadden, Conditional logit analysis of qualitative choice behavior, in P. Zarembka (ed.), Frontiers in Econometrics, Academic Press, New York, 1974, pp. 105–142.
  • W. Feller, An Introduction to Probability Theory and Its Applications, Vol. II, Wiley, 1966 (Lindeberg–Feller theorem, pp. 256–258).
  • C. R. Rao, Linear Statistical Inference and Its Applications, Wiley (cited by McFadden as Rao (1968), pp. 347–351, for the asymptotic χ2\chi^2χ2 test).
10 thms2 active usersReviewed
🏆Completed
Algorithmic Game TheoryComplexity TheoryOperations Research·Captain: mikedeng1

The Complexity of Computing a Nash Equilibrium 3: Trimming an Approximate Nash Equilibrium Yields a Well-Supported OneResearch Paper

Motivation

The complexity of computing a Nash equilibrium is usually studied for an approximate equilibrium, because an exact equilibrium of a game with three or more players can require irrational probabilities. Two approximation notions are in use, and results proved for one do not automatically transfer to the other. Daskalakis, Goldberg and Papadimitriou (SIAM J. Comput. 39(1), 2009) prove their PPAD-hardness results for the stronger notion, the ε-approximately well-supported Nash equilibrium, and then show in their §4.7 (Lemma 4.28, announced as Lemma 2.1) that any ε-approximate Nash equilibrium can be turned, by an explicit and cheap transformation, into an approximately well-supported one with a polynomially worse accuracy. This is what lets their hardness results hold for both notions. A weaker version of the relation had been pointed out by Chen, Deng and Teng (FOCS 2006, reference [9] of the paper).

This mission formalizes Lemma 4.28 together with the steps of its proof: the best-response inequality (28), Claims 5 and 6, and the Lipschitz estimate of Lemma 4.26 (used in the proof as Lemma 4.29).

Setting

A game in normal form has r≥2r\ge2r≥2 players. Player ppp has a finite set SpS_pSp​ of pure strategies; S=∏pSpS=\prod_p S_pS=∏p​Sp​ is the set of pure strategy profiles and S−pS_{-p}S−p​ the set of profiles of the players other than ppp. For each player ppp and profile s∈Ss\in Ss∈S there is a payoff usp≥0u^p_s\ge0usp​≥0; for j∈Spj\in S_pj∈Sp​ and s∈S−ps\in S_{-p}s∈S−p​ the payoff to ppp at the profile (j,s)(j,s)(j,s) is written ujspu^p_{js}ujsp​. The number max⁡{u}\max\{u\}max{u} is the largest entry uspu^p_susp​ over all players and all profiles.

A mixed profile x={xjp}x=\{x^p_j\}x={xjp​} assigns to each player ppp a probability distribution xpx^pxp on SpS_pSp​; the players randomize independently, and for s∈S−ps\in S_{-p}s∈S−p​ we write xs=∏q≠pxsqqx_s=\prod_{q\ne p}x^q_{s_q}xs​=∏q=p​xsq​q​. The expected payoff of player ppp for playing the pure strategy jjj against the others is

Ujp=∑s∈S−pujsp xs,Umax⁡p=max⁡j∈SpUjp.\mathcal U^p_j=\sum_{s\in S_{-p}}u^p_{js}\,x_s,\qquad \mathcal U^p_{\max}=\max_{j\in S_p}\mathcal U^p_j .Ujp​=s∈S−p​∑​ujsp​xs​,Umaxp​=j∈Sp​max​Ujp​.
  • xxx is an ε-approximate Nash equilibrium if no player can gain more than ϵ\epsilonϵ by deviating to any mixed strategy ypy^pyp:
∑j∈SpUjp xjp ≥ ∑j∈SpUjp yjp−ϵfor all p and all mixed yp.\sum_{j\in S_p}\mathcal U^p_j\,x^p_j\ \ge\ \sum_{j\in S_p}\mathcal U^p_j\,y^p_j-\epsilon\quad\text{for all }p\text{ and all mixed }y^p .j∈Sp​∑​Ujp​xjp​ ≥ j∈Sp​∑​Ujp​yjp​−ϵfor all p and all mixed yp.
  • xxx is an ε-approximately well-supported Nash equilibrium if every strategy played with positive probability is within ϵ\epsilonϵ of the best:
Ujp>Uj′p+ϵ ⟹ xj′p=0for all p and j,j′∈Sp.\mathcal U^p_j>\mathcal U^p_{j'}+\epsilon\ \Longrightarrow\ x^p_{j'}=0\quad\text{for all }p\text{ and }j,j'\in S_p .Ujp​>Uj′p​+ϵ ⟹ xj′p​=0for all p and j,j′∈Sp​.

The trimmed profile x^\hat xx^ of xxx with parameter kkk deletes the strategies whose expected payoff is more than ϵk\epsilon kϵk below Umax⁡p\mathcal U^p_{\max}Umaxp​ and renormalizes:

zp=∑j∈Spxjp X{Ujp<Umax⁡p−ϵk},x^jp={xjp/(1−zp),Ujp≥Umax⁡p−ϵk,0,otherwise.z^p=\sum_{j\in S_p}x^p_j\,\mathcal X_{\{\mathcal U^p_j<\mathcal U^p_{\max}-\epsilon k\}},\qquad \hat x^p_j=\begin{cases}x^p_j/(1-z^p), & \mathcal U^p_j\ge\mathcal U^p_{\max}-\epsilon k,\\ 0, & \text{otherwise.}\end{cases}zp=j∈Sp​∑​xjp​X{Ujp​<Umaxp​−ϵk}​,x^jp​={xjp​/(1−zp),0,​Ujp​≥Umaxp​−ϵk,otherwise.​

In the Lean development these are purePayoff u x p j, maxPurePayoff u x p, maxPayoff u, IsEpsApproxNash u x ε, IsEpsWellSupportedNash u x ε, trimMass u x ε k p and trim u x ε k, all in the namespace DGPNash.WellSupported.

Formalization targets

Goal: Lemma 4.28

For ϵ>0\epsilon>0ϵ>0 and an ϵ\epsilonϵ-approximate Nash equilibrium xxx, the trimmed profile x^\hat xx^ with k=1+1/ϵk=1+1/\sqrt\epsilonk=1+1/ϵ​ is a mixed profile and a

ϵ⋅(ϵ+1+4(r−1)max⁡{u})-approximately well-supported Nash equilibrium.\sqrt\epsilon\cdot\bigl(\sqrt\epsilon+1+4(r-1)\max\{u\}\bigr)\text{-approximately well-supported Nash equilibrium.}ϵ​⋅(ϵ​+1+4(r−1)max{u})-approximately well-supported Nash equilibrium.

The constant is the paper's. The statement is about the specific profile x^\hat xx^ built from xxx, not about the existence of some well-supported equilibrium.

Milestones, in proof order

  • Eq. (28): ∑jUjpxjp≥Umax⁡p−ϵ\sum_j\mathcal U^p_j x^p_j\ge\mathcal U^p_{\max}-\epsilon∑j​Ujp​xjp​≥Umaxp​−ϵ for every player ppp.
  • Claim 5: for every k>0k>0k>0, zp≤1/kz^p\le 1/kzp≤1/k.
  • Claim 6: for every k>1k>1k>1, ∑j∈Sp∣xjp−x^jp∣≤2/(k−1)\sum_{j\in S_p}|x^p_j-\hat x^p_j|\le 2/(k-1)∑j∈Sp​​∣xjp​−x^jp​∣≤2/(k−1).
  • Lemma 4.26 (Lemma 4.29): for mixed profiles x,yx,yx,y,
∣∑s∈S−pujspxs−∑s∈S−pujspys∣≤max⁡s∈S−p{ujsp}∑q≠p∑i∈Sq∣xiq−yiq∣.\Bigl|\sum_{s\in S_{-p}}u^p_{js}x_s-\sum_{s\in S_{-p}}u^p_{js}y_s\Bigr|\le\max_{s\in S_{-p}}\{u^p_{js}\}\sum_{q\ne p}\sum_{i\in S_q}|x^q_i-y^q_i| .​s∈S−p​∑​ujsp​xs​−s∈S−p​∑​ujsp​ys​​≤s∈S−p​max​{ujsp​}q=p∑​i∈Sq​∑​∣xiq​−yiq​∣.

Significance

The result. Lemma 4.28 makes the two approximation notions polynomially equivalent for computation: every well-supported equilibrium is approximate with the same ϵ\epsilonϵ, and conversely an approximate equilibrium yields a well-supported one at accuracy O(ϵ)O(\sqrt\epsilon)O(ϵ​) for fixed rrr and payoff range. Hardness results proved for well-supported equilibria, including the PPAD-completeness of 3-player Nash in the paper and the two-player result of Chen, Deng and Teng, therefore apply to approximate equilibria as well, and algorithms for one notion give algorithms for the other. Lemma 4.26, the Lipschitz dependence of expected payoffs on the opponents' strategies in L1L_1L1​, is a general tool for perturbation and rounding arguments in finite games.

Formalizing it. The result is proved in the paper; as far as we know there is no machine-checked version, and the platform has neither approximation notion. The mission produces both definitions, on top of the published game vocabulary agt_games, and the quantitative chain from an approximate equilibrium to a well-supported one. These definitions are what any later formalization of approximate-equilibrium algorithms or hardness results would need.

Difficulty

The obvious attempt, to show that xxx itself is approximately well-supported, fails: an approximate equilibrium may put small positive probability on a strategy that is far from optimal, which a well-supported equilibrium forbids for any accuracy. Removing those strategies changes the opponents' expected payoffs, so the well-supported condition must be checked for U^\hat{\mathcal U}U^, computed from x^\hat xx^, not for U\mathcal UU. The accuracy of the result therefore depends on how far all r−1r-1r−1 opponents' strategies move, which is why the bound carries the factor (r−1)max⁡{u}(r-1)\max\{u\}(r−1)max{u}. Lemma 4.26 is a statement about product distributions: the change in a multilinear expected payoff is controlled by the sum of the per-player L1L_1L1​ distances, not by their product.

Formalization scope

  • Players form a finite type ι with decidable equality; player p has a finite strategy type S p; payoffs are u : ι → (∀ i, S i) → ℝ with the standing hypothesis ∀ p s, 0 ≤ u p s of §2.1. The number of players is r=r=r= Fintype.card ι, and 2 ≤ Fintype.card ι is assumed in every statement, as in §2.1. Strategy sets may differ between players.
  • Mixed profiles and lotteries are AGT.IsMixedProfile and AGT.IsLottery from agt_games; both equilibrium notions include the requirement that the profile be mixed. Ujp\mathcal U^p_jUjp​ is AGT.expectedPayoff after player ppp alone switches to the pure strategy jjj, and the approximate-Nash condition compares AGT.expectedPayoff before and after player ppp alone switches to a lottery yyy, which is the paper's inequality (27).
  • Umax⁡p\mathcal U^p_{\max}Umaxp​ and max⁡{u}\max\{u\}max{u} are suprema over finite types; they are the maxima, since a mixed profile forces each SpS_pSp​ to be nonempty. In Lemma 4.26 the maximum over s∈S−ps\in S_{-p}s∈S−p​ is the supremum of upu^pup over full profiles whose ppp-th coordinate is jjj.
  • The strict and non-strict inequalities are the paper's: the trim keeps Ujp≥Umax⁡p−ϵk\mathcal U^p_j\ge\mathcal U^p_{\max}-\epsilon kUjp​≥Umaxp​−ϵk, zpz^pzp counts Ujp<Umax⁡p−ϵk\mathcal U^p_j<\mathcal U^p_{\max}-\epsilon kUjp​<Umaxp​−ϵk, and the well-supported condition uses >>>. The goal assumes ϵ>0\epsilon>0ϵ>0, as the paper's 1/ϵ1/\sqrt\epsilon1/ϵ​ requires; Claim 5 is stated for k>0k>0k>0 and Claim 6 for k>1k>1k>1.
  • The clause "can be computed in polynomial time" of Lemma 4.28 is not formalized; the goal states the property of the profile the proof computes. A complexity statement would need the PPAD machinery, which is outside this series.
  • A trivializing formalization is ruled out: by Nash's theorem (on the platform as AGT.nash_existence) a δ-well-supported equilibrium exists for every δ ≥ 0, so the goal is stated for x^\hat xx^, defined from xxx, and not as an existence claim.

Contributions welcome: proofs of the milestones, in particular the decomposition ∑sxsusp=∑jxjp Ujp\sum_s x_s u^p_s=\sum_j x^p_j\,\mathcal U^p_j∑s​xs​usp​=∑j​xjp​Ujp​ of the expected payoff, which Eq. (28) and the goal both need, and Lemma 4.26, which is reusable in any perturbation argument for finite games.

Selected references

  • C. Daskalakis, P. W. Goldberg, C. H. Papadimitriou, The Complexity of Computing a Nash Equilibrium, SIAM Journal on Computing 39(1):195–259, 2009. https://doi.org/10.1137/070699652
  • X. Chen, X. Deng, S.-H. Teng, Computing Nash Equilibria: Approximation and Smoothed Complexity, FOCS 2006. https://arxiv.org/abs/cs/0602043
  • X. Chen, X. Deng, S.-H. Teng, Settling the Complexity of Computing Two-Player Nash Equilibria, Journal of the ACM 56(3), 2009. https://doi.org/10.1145/1516512.1516516
  • J. Nash, Non-Cooperative Games, Annals of Mathematics 54(2):286–295, 1951. https://doi.org/10.2307/1969529
8 thms4 active usersReviewed
🏆Completed
Operations ResearchOptimization·Captain: mikedeng1

One-Machine Sequencing to Minimize Certain Functions of Job Tardiness II: EDD Order Minimizes Any Sum of Convex Nondecreasing Tardiness Penalties When No Job Starts After Its Due DateResearch Paper

Motivation

A single machine must process a set of jobs, each with a processing time and a due date, and the cost of a schedule depends on how late the jobs finish. Total tardiness is the classical criterion, but in many applications lateness is penalised more than proportionally: a job one week late costs more than twice a job half a week late, and a quadratic or other convex penalty describes this better. Hamilton Emmons's 1969 paper in Operations Research (DOI 10.1287/opre.17.4.701) derives dominance rules for total tardiness and then asks which of them survive when total tardiness is replaced by ∑Jg(Ti)\sum_J g(T_i)∑J​g(Ti​) for an arbitrary convex nondecreasing loss ggg.

Timeline of the relevant results:

  • 1955 — Jackson shows that ordering jobs by earliest due date (EDD) minimises the maximum lateness, and hence produces a schedule without late jobs whenever one exists.
  • 1956 — Smith gives the ratio rule for weighted completion time and an adjacent-interchange criterion for pairs of jobs.
  • 1969 — Emmons proves the precedence theorems for total tardiness that underlie later branch-and-bound and dynamic programming algorithms for 1 ∣∣ ∑Tj1\,||\,\sum T_j1∣∣∑Tj​, and shows (p. 713) that Theorems 2 and 3 and part of Theorem 1 extend to any sum of identical convex nondecreasing tardiness penalties.
  • 1977 — Lawler's pseudo-polynomial algorithm for total tardiness builds on Emmons's conditions; the problem is later shown NP-hard (Du and Leung, 1990).

Setting

A finite set JJJ of jobs is to be sequenced on one machine. Job JiJ_iJi​ has a processing time pi≥0p_i\ge 0pi​≥0 and a due date did_idi​. All jobs are available at time 000, and the machine processes them one after another without idle time. A schedule is an ordering lll of the jobs of JJJ. The completion time CiC_iCi​ of JiJ_iJi​ in lll is the sum of the processing times of JiJ_iJi​ and of every job before it; its waiting (starting) time is Wi=Ci−piW_i=C_i-p_iWi​=Ci​−pi​, and its tardiness is

Ti=max⁡(0, Ci−di).T_i=\max(0,\,C_i-d_i).Ti​=max(0,Ci​−di​).

A loss function g:R→Rg:\mathbb R\to\mathbb Rg:R→R, convex and nondecreasing on [0,∞)[0,\infty)[0,∞), is fixed, the same for every job. The objective is ∑i∈Jg(Ti)\sum_{i\in J} g(T_i)∑i∈J​g(Ti​), and a schedule is optimal if no schedule of JJJ has a smaller objective. With g(T)=Tg(T)=Tg(T)=T this is total tardiness.

Where the paper uses job indices, the jobs are indexed in SPT order: j<kj<kj<k implies pj<pkp_j<p_kpj​<pk​, or pj=pkp_j=p_kpj​=pk​ and dj≤dkd_j\le d_kdj​≤dk​. The notation j←kj\leftarrow kj←k means that some optimal schedule has JjJ_jJj​ before JkJ_kJk​; for a set AkA_kAk​ of jobs, k←Akk\leftarrow A_kk←Ak​ means that some optimal schedule has JkJ_kJk​ before every job of AkA_kAk​, and Ak′A_k'Ak′​ is the set of jobs of JJJ not in AkA_kAk​. An EDD schedule sequences the jobs in nondecreasing order of due dates.

Formalization targets

Goal: Corollary 2.2* (p. 713)

If an EDD schedule lll of JJJ satisfies

Wi≤difor every i∈J,W_i\le d_i\qquad\text{for every } i\in J,Wi​≤di​for every i∈J,

then lll minimises ∑Jg(Ti)\sum_J g(T_i)∑J​g(Ti​) over all schedules of JJJ, for every ggg convex and nondecreasing on [0,∞)[0,\infty)[0,∞).

The goal fixes no constant and no particular ggg: it is a statement about the whole class of convex nondecreasing penalties.

Milestones (in attack order)

  1. Convex exchange condition (p. 713): if Tja≤TjbT_{ja}\le T_{jb}Tja​≤Tjb​, Tkb≤TkaT_{kb}\le T_{ka}Tkb​≤Tka​ (all nonnegative), Tka−Tkb≤Tjb−TjaT_{ka}-T_{kb}\le T_{jb}-T_{ja}Tka​−Tkb​≤Tjb​−Tja​ and Tjb≥TkaT_{jb}\ge T_{ka}Tjb​≥Tka​, then g(Tka)−g(Tkb)≤g(Tjb)−g(Tja)g(T_{ka})-g(T_{kb})\le g(T_{jb})-g(T_{ja})g(Tka​)−g(Tkb​)≤g(Tjb​)−g(Tja​).
  2. Theorem 1* (p. 713): for j<kj<kj<k, if dj≤dkd_j\le d_kdj​≤dk​ then j←kj\leftarrow kj←k.
  3. Theorem 2* (p. 713): for j<kj<kj<k, if k←Akk\leftarrow A_kk←Ak​, dj>dkd_j>d_kdj​>dk​ and dj+pj≥∑Ak′pid_j+p_j\ge\sum_{A_k'}p_idj​+pj​≥∑Ak′​​pi​, then k←jk\leftarrow jk←j.
  4. Corollary 2.1* (p. 713): if dj=max⁡idid_j=\max_i d_idj​=maxi​di​ and dj+pj≥∑Jpid_j+p_j\ge\sum_J p_idj​+pj​≥∑J​pi​, then some optimal schedule ends with JjJ_jJj​.
  5. Last-job reduction (proof of Corollary 2.2, p. 706): if some optimal schedule ends with JjJ_jJj​, any optimal schedule of J∖{Jj}J\setminus\{J_j\}J∖{Jj​} followed by JjJ_jJj​ is optimal for JJJ.

Two further statements of the same section are included as items: Corollary 1.3* (the SPT schedule is optimal if it coincides with the EDD schedule) and Theorem 3 for the generalised objective.

Significance

The goal says that EDD is optimal for every convex nondecreasing tardiness penalty as long as no job starts after its due date. The classical sufficient condition, that at most one job is tardy, follows from Jackson's rule; Emmons's condition allows any or all jobs to be tardy, provided each is tardy by at most its own processing time. Because the conclusion holds for the whole class of penalties at once, an instance satisfying it needs no knowledge of ggg: total tardiness, total squared tardiness, and any other convex nondecreasing cost are minimised by the same sequence. Theorems 1* and 2* are the dominance rules that the paper's ordering procedure applies pairwise; they reduce the search space of branch-and-bound methods for convex tardiness objectives.

The results are proved in the paper (for ∑g(Ti)\sum g(T_i)∑g(Ti​) the proofs are said to be "easily established" and omitted). None of them has, to our knowledge, a machine-checked proof. The mission produces formal statements and proofs of the generalised results, including the omitted ones, on top of a reusable single-machine model.

Difficulty

The total-tardiness proofs compare changes in tardiness additively: an interchange is good if the decrease in one job's tardiness is at least the increase in another's. For a convex ggg this comparison is not enough, because a unit of tardiness costs more at higher tardiness levels; the changes must also occur at the right height on the curve, and part (b) of the proof of Theorem 1 fails for this reason. Each generalised argument therefore has to check, for every job whose tardiness changes, both the size and the location of the change, including jobs whose tardiness changes from zero to positive. Ties for the latest due date are a further obstacle: the printed proof of Corollary 2.1 cites Theorem 2, whose hypothesis dj>dkd_j>d_kdj​>dk​ is strict, so a job sharing the maximum due date is not covered by the argument as written, although the corollary is stated without excluding ties.

Formalization scope

Jobs are elements of a type ι\iotaι; the job set is a Finset JJJ, processing times and due dates are real functions p,d:ι→Rp,d:\iota\to\mathbb Rp,d:ι→R. Where the paper's index matters, ι\iotaι is linearly ordered and its order is the job index, together with the SPT-indexing hypothesis. A schedule is a duplicate-free list whose elements are exactly JJJ, and completion times are the published single-machine definition MooreLateJobs.Shared.completionTime (Moore 1968), which starts the machine at time 000 with no idle time. Optimality is against every schedule of JJJ. The relation j←kj\leftarrow kj←k is formalised as the existence of an optimal schedule with JjJ_jJj​ before JkJ_kJk​ (keeping the premise k←Akk\leftarrow A_kk←Ak​ in the conclusion where the theorem has one); the paper's cumulative reading of the notation is not formalised.

Standing assumptions and deviations:

  • ggg is convex and nondecreasing on [0,∞)[0,\infty)[0,∞) only; the page's "increasing" is read as nondecreasing, as in the abstract. No smoothness, strict monotonicity or g(0)=0g(0)=0g(0)=0 is assumed.
  • Processing times are assumed nonnegative; this is added (they are durations).
  • The reduction di<∑Jpid_i<\sum_J p_idi​<∑J​pi​ of p. 703 is not assumed, which makes the statements apply to more instances.
  • "The EDD schedule" is any schedule with nondecreasing due dates; ties are arbitrary.

A trivializing formalization is ruled out: the goal requires optimality of the given EDD list against every schedule of JJJ, not of some EDD list, and a sorry-free check shows its hypotheses hold on a two-job instance in which both jobs are tardy.

A complete development needs list lemmas for moving one job to a later position, the effect of such moves on completion times, and slope inequalities for convex functions on [0,∞)[0,\infty)[0,∞). The schedule-manipulation lemmas are reusable for other single-machine sequencing results. Proofs of any milestone, of the two further items, and of general interchange lemmas are welcome.

Selected references

  • H. Emmons, One-Machine Sequencing to Minimize Certain Functions of Job Tardiness, Operations Research 17(4):701–715, 1969. https://doi.org/10.1287/opre.17.4.701
  • J. R. Jackson, Scheduling a Production Line to Minimize Maximum Tardiness, Research Report 43, Management Science Research Project, UCLA, 1955.
  • W. E. Smith, Various Optimizers for Single-Stage Production, Naval Research Logistics Quarterly 3:59–66, 1956. https://doi.org/10.1002/nav.3800030106
  • E. L. Lawler, A "Pseudopolynomial" Algorithm for Sequencing Jobs to Minimize Total Tardiness, Annals of Discrete Mathematics 1:331–342, 1977. https://doi.org/10.1016/S0167-5060(08)70742-8
  • J. M. Moore, An n Job, One Machine Sequencing Algorithm for Minimizing the Number of Late Jobs, Management Science 15(1):102–109, 1968. https://doi.org/10.1287/mnsc.15.1.102
  • J. Du and J. Y.-T. Leung, Minimizing Total Tardiness on One Machine is NP-Hard, Mathematics of Operations Research 15(3):483–495, 1990. https://doi.org/10.1287/moor.15.3.483
9 thms3 active usersReviewed
🏆Completed
Algorithmic Game TheoryComplexity TheoryOperations Research·Captain: mikedeng1

The Complexity of Computing a Nash Equilibrium 2: The Three-Colorable Addition-Multiplication Gadget Has Error Amplification 81Research Paper

Motivation

Nash's theorem guarantees that every finite game has a mixed equilibrium, but says nothing about how hard one is to find. Daskalakis, Goldberg and Papadimitriou (SIAM J. Comput. 39(1), 2009) showed that computing a Nash equilibrium is complete for the class PPAD of total search problems whose solutions are guaranteed by a parity argument on a directed graph (Papadimitriou 1994). Their proof simulates an arithmetic circuit by a graphical game: a multiplayer game in which each player's payoff depends on a few neighbours only, built from small game gadgets whose equilibria compute x↦αxx \mapsto \alpha xx↦αx, x+yx + yx+y, xyxyxy, constants and comparisons on mixed-strategy probabilities.

These gadgets are the reusable core of the reduction. The PPAD-completeness results for three-player games, and the later results for two-player games and for approximate equilibria, all rest on gadget analyses of this kind.

Timeline.

  • 1994: Papadimitriou defines PPAD and shows that Nash equilibrium computation lies in it (JCSS 48).
  • 2001: Kearns, Littman and Singh introduce graphical games (UAI 2001).
  • 2005–2006: Goldberg and Papadimitriou reduce rrr-player games to four-player games, and Daskalakis, Goldberg and Papadimitriou prove PPAD-completeness for four players (STOC 2006). The three-player case is obtained independently by Daskalakis and Papadimitriou and by Chen and Deng (ECCC reports, 2005); the two-player case by Chen and Deng (FOCS 2006; J. ACM 56(3), 2009, with Teng).
  • 2009: the journal version unifies the three-player argument; its Proposition 4.18 is the gadget G+,∗\mathcal G_{+,*}G+,∗​ that makes the reduction to three players work.

Setting

A game has finitely many players; player ppp has a finite set SpS_pSp​ of pure strategies and a nonnegative payoff uspu^p_susp​ for each pure profile sss. A mixed profile xxx gives each player a probability distribution (xjp)j∈Sp(x^p_j)_{j\in S_p}(xjp​)j∈Sp​​, and xsx_sxs​ denotes the product of the probabilities of the components of a partial profile sss. The expected payoff of a pure strategy jjj of ppp is ∑s∈S−pujspxs\sum_{s\in S_{-p}} u^p_{js}x_s∑s∈S−p​​ujsp​xs​.

An ϵ\epsilonϵ-Nash equilibrium (the paper's notion, an ϵ\epsilonϵ-approximately well-supported equilibrium, Eq. (2)) is a mixed profile in which no player puts weight on a pure strategy that another of its pure strategies beats by more than ϵ\epsilonϵ:

∀p, j,j′∈Sp:∑s∈S−pujspxs>∑s∈S−puj′spxs+ϵ  ⟹  xj′p=0.\forall p,\ j, j' \in S_p:\quad \sum_{s\in S_{-p}} u^p_{js}x_s > \sum_{s\in S_{-p}} u^p_{j's}x_s + \epsilon \implies x^p_{j'} = 0 .∀p, j,j′∈Sp​:s∈S−p​∑​ujsp​xs​>s∈S−p​∑​uj′sp​xs​+ϵ⟹xj′p​=0.

For ϵ=0\epsilon = 0ϵ=0 this characterizes Nash equilibria.

A player with strategies {0,1}\{0,1\}{0,1} is binary; p[v]\mathbf p[v]p[v] is the probability that it plays 111, and p[v:s]\mathbf p[v : s]p[v:s] is the probability that vvv plays sss. The affects graph (Definition 2.2) has an edge (v1,v2)(v_1, v_2)(v1​,v2​) between distinct players when the payoff of v2v_2v2​ is a nonconstant function of the action of v1v_1v1​. A legal kkk-coloring (Definition 4.8) gives each player one of kkk colors so that the two ends of an edge differ and two distinct players with a common successor differ.

The game G+,∗\mathcal G_{+,*}G+,∗​ (Fig. 11) has inputs v1,v2v_1, v_2v1​,v2​, output v3v_3v3​ and intermediate players w1,v1′,w2,v2′,w3,w,uw_1, v_1', w_2, v_2', w_3, w, uw1​,v1′​,w2​,v2′​,w3​,w,u, all binary except v2′v_2'v2′​, which has strategies {0,1,∗}\{0, 1, *\}{0,1,∗}. Its payoff tables are fixed by nonnegative integers α,β,γ\alpha, \beta, \gammaα,β,γ; the inputs' payoffs are unconstrained.

Formalization targets

Goal: Proposition 4.18

For nonnegative integers α,β,γ\alpha, \beta, \gammaα,β,γ with α+β+γ≤3\alpha + \beta + \gamma \le 3α+β+γ≤3, the affects graph of G+,∗\mathcal G_{+,*}G+,∗​ has a legal 3-coloring, and for every ϵ∈[0,0.01]\epsilon \in [0, 0.01]ϵ∈[0,0.01], at every ϵ\epsilonϵ-Nash equilibrium,

∣ p[v3]−min⁡{1, αp[v1]+βp[v2]+γp[v1]p[v2]}∣≤81 ϵ.\Big|\,\mathbf p[v_3] - \min\{1,\ \alpha\mathbf p[v_1] + \beta\mathbf p[v_2] + \gamma\mathbf p[v_1]\mathbf p[v_2]\}\Big| \le 81\,\epsilon .​p[v3​]−min{1, αp[v1​]+βp[v2​]+γp[v1​]p[v2​]}​≤81ϵ.

The case ϵ=0\epsilon = 0ϵ=0 is exact computation at every Nash equilibrium.

Milestones: the four claims of the proof

  • Claim 1: p[v1′]=18p[v1]±ϵ\mathbf p[v_1'] = \tfrac18\mathbf p[v_1] \pm \epsilonp[v1′​]=81​p[v1​]±ϵ.
  • Claim 2: p[v2′:1]=18p[v2]±ϵ\mathbf p[v_2' : 1] = \tfrac18\mathbf p[v_2] \pm \epsilonp[v2′​:1]=81​p[v2​]±ϵ.
  • Claim 3: p[v2′:∗]=α8p[v1]+β8p[v2]+γ8p[v1]p[v2]±10ϵ\mathbf p[v_2' : *] = \tfrac{\alpha}{8}\mathbf p[v_1] + \tfrac{\beta}{8}\mathbf p[v_2] + \tfrac{\gamma}{8}\mathbf p[v_1]\mathbf p[v_2] \pm 10\epsilonp[v2′​:∗]=8α​p[v1​]+8β​p[v2​]+8γ​p[v1​]p[v2​]±10ϵ.
  • Claim 4: the output bound of the goal.

Milestones: the elementary gadgets

  • Proposition 4.2 (G×α\mathcal G_{\times\alpha}G×α​): p[v2]=min⁡(αp[v1],1)±ϵ\mathbf p[v_2] = \min(\alpha\mathbf p[v_1], 1) \pm \epsilonp[v2​]=min(αp[v1​],1)±ϵ for real α≥0\alpha \ge 0α≥0, ϵ<1\epsilon < 1ϵ<1.
  • Proposition 4.3: p[v3]=min⁡(αp[v1]+βp[v2]+γp[v1]p[v2],1)±ϵ\mathbf p[v_3] = \min(\alpha\mathbf p[v_1] + \beta\mathbf p[v_2] + \gamma\mathbf p[v_1]\mathbf p[v_2], 1) \pm \epsilonp[v3​]=min(αp[v1​]+βp[v2​]+γp[v1​]p[v2​],1)±ϵ for real α,β,γ≥0\alpha, \beta, \gamma \ge 0α,β,γ≥0.
  • Proposition 4.5 (Gα\mathcal G_\alphaGα​): p[v1]=min⁡(α,1)±ϵ\mathbf p[v_1] = \min(\alpha, 1) \pm \epsilonp[v1​]=min(α,1)±ϵ.
  • Lemma 5.3 (comparator G<\mathcal G_<G<​): p[d]=1\mathbf p[d] = 1p[d]=1 if p[a]<p[b]−ϵ\mathbf p[a] < \mathbf p[b] - \epsilonp[a]<p[b]−ϵ and p[d]=0\mathbf p[d] = 0p[d]=0 if p[a]>p[b]+ϵ\mathbf p[a] > \mathbf p[b] + \epsilonp[a]>p[b]+ϵ.

Significance

The result. The paper reduces rrr-player games to three-player games by building a graphical game from gadgets, legally coloring it with three colors, and turning each color class into one player of a normal-form game (§4.2, §4.5). Proposition 4.18 supplies a gadget that computes min⁡{1,αx+βy+γxy}\min\{1, \alpha x + \beta y + \gamma xy\}min{1,αx+βy+γxy} and can be glued into such a construction while keeping it legally 3-colorable (p. 233), at the price of an error constant 818181 instead of the constant 111 of the elementary gadgets. The error bound is what carries the reduction over to approximate equilibria: an ϵ\epsilonϵ-Nash equilibrium of the gadget computes its function to within 81ϵ81\epsilon81ϵ.

Formalizing it. All results here are proved in the paper (Proposition 4.5 is stated with the remark that its proof is similar to those of Propositions 4.2 and 4.3). None of them has a machine-checked proof, and no graphical-game or gadget infrastructure exists in Lean. The mission formalizes the known proofs and produces a library of verified gadgets with explicit error bounds, the first layer of any formal treatment of the PPAD-hardness of Nash equilibria.

Difficulty

Each claim is a case analysis on which strategy of an intermediate player is dominated by more than ϵ\epsilonϵ, but the cases interact. Claim 3 needs Claims 1 and 2 as inputs, and it is the step where both α+β+γ≤3\alpha + \beta + \gamma \le 3α+β+γ≤3 and ϵ≤0.01\epsilon \le 0.01ϵ≤0.01 are used: without them one of the regimes of www cannot be excluded. The three-strategy player v2′v_2'v2′​ cannot be handled by the "binary player copies or disagrees" argument of the elementary gadgets; its weight on 111 and on ∗*∗ must be tracked separately. The error constants 101010 and 818181 propagate through products of approximate quantities, so a proof that is loose at any step does not reach them.

On the Lean side, the expected payoff of a pure strategy is a sum over all pure profiles of a ten-player game; every claim first has to reduce it to the few coordinates the player's payoff actually depends on.

Formalization scope

Games, mixed profiles and expected payoffs are those of the published agt_games bundle: a finite player type, strategy types S i, payoffs u : ι → (∀ i, S i) → ℝ, AGT.IsMixedProfile, AGT.expectedPayoff. The expected payoff of a pure strategy jjj is the expected payoff of the profile with ppp's strategy replaced by the point mass at jjj. An ϵ\epsilonϵ-Nash equilibrium requires a genuine mixed profile.

Each gadget is one explicit game on its own player type, with the payoff tables of the paper. Binary strategies are Fin 2 with the paper's labels 0,10, 10,1, and p[v]\mathbf p[v]p[v] is the weight on 1; v2′v_2'v2′​'s strategies are an inductive type {zero, one, star}. The input players' payoffs are unconstrained in the paper; here they are 000, which makes the ϵ\epsilonϵ-condition at the inputs vacuous. Since every non-input payoff depends only on gadget players, "ϵ\epsilonϵ-Nash equilibrium of the standalone gadget" quantifies over all mixed strategies of the inputs and imposes the condition exactly on the other players, which is the paper's meaning when the gadget sits inside a larger game. The affects graph is computed from the payoff functions, without self-loops, and colors {1,…,k}\{1,\dots,k\}{1,…,k} are Fin k. The ranges α,β,γ∈N\alpha, \beta, \gamma \in \mathbb Nα,β,γ∈N (Proposition 4.18) and α,β,γ∈R≥0\alpha, \beta, \gamma \in \mathbb R_{\ge 0}α,β,γ∈R≥0​ (Propositions 4.2, 4.3, 4.5) follow the paper; ϵ≥0\epsilon \ge 0ϵ≥0 is added where the paper writes only ϵ<1\epsilon < 1ϵ<1.

The paper states Proposition 4.18 and Lemma 5.3 as "there is a graphical game"; an existential statement would be met by a game whose output has a dominant strategy and whose inputs are forced, so both are stated for the explicit game of the proof, with free inputs.

A complete development needs a computation lemma for the pure-strategy payoff in each gadget (reusable for any gadget on these definitions) and the case analyses of the claims. Proofs of the elementary gadgets, which are short, and of Claims 1 and 2 are good first contributions; Claim 3 is the main step.

Selected references

  • C. Daskalakis, P. W. Goldberg, C. H. Papadimitriou, The Complexity of Computing a Nash Equilibrium, SIAM J. Comput. 39(1):195–259, 2009. https://doi.org/10.1137/070699652
  • C. H. Papadimitriou, On the complexity of the parity argument and other inefficient proofs of existence, J. Comput. System Sci. 48(3):498–532, 1994. https://doi.org/10.1016/S0022-0000(05)80063-7
  • M. Kearns, M. L. Littman, S. Singh, Graphical Models for Game Theory, UAI 2001. https://arxiv.org/abs/1301.2281
  • X. Chen, X. Deng, S.-H. Teng, Settling the complexity of computing two-player Nash equilibria, J. ACM 56(3):14, 2009. https://doi.org/10.1145/1516512.1516516
15 thms3 active usersReviewed
AnalysisConvex OptimizationOperations Research+1·Captain: mikedeng1

The Łojasiewicz Inequality for Nonsmooth Subanalytic Functions with Applications to Subgradient Dynamical Systems II: The Łojasiewicz Inequality for Convex Subanalytic Functions on Bounded SetsResearch Paper

Motivation

The Łojasiewicz inequality states that near a critical point aaa of a real-analytic function fff there are θ∈[0,1)\theta\in[0,1)θ∈[0,1) and CCC with ∣f(x)−f(a)∣θ≤C ∥∇f(x)∥|f(x)-f(a)|^{\theta}\le C\,\|\nabla f(x)\|∣f(x)−f(a)∣θ≤C∥∇f(x)∥. Łojasiewicz used it in the 1960s to prove that every bounded trajectory of the gradient flow x˙=−∇f(x)\dot x=-\nabla f(x)x˙=−∇f(x) has finite length and converges to a single critical point, a conclusion that fails for general C∞C^\inftyC∞ functions. The inequality has since become the standard tool for convergence analysis of descent methods on nonconvex problems.

Optimization problems are, however, rarely smooth: constraints enter through indicator functions, and objectives contain norms, maxima and penalties. Bolte, Daniilidis and Lewis (SIAM J. Optim. 17 (2007) 1205–1223) extended the inequality to nonsmooth subanalytic functions, replacing ∥∇f∥\|\nabla f\|∥∇f∥ by a slope built from the limiting subdifferential. Their Section 3.1 treats functions continuous on a closed domain; Section 3.2, the subject of this mission, treats lower semicontinuous convex functions, which may jump to +∞+\infty+∞ and whose domain need not be closed. The Kurdyka–Łojasiewicz framework built on this paper (Attouch–Bolte–Svaiter 2013; Bolte–Sabach–Teboulle 2014) underlies the convergence theory of proximal and splitting algorithms used throughout operations research.

Setting

Work in Rn\mathbb R^nRn with the Euclidean norm, and let f:Rn→R∪{+∞}f:\mathbb R^n\to\mathbb R\cup\{+\infty\}f:Rn→R∪{+∞} with domain dom⁡f={x:f(x)<+∞}\operatorname{dom} f=\{x: f(x)<+\infty\}domf={x:f(x)<+∞}.

A set A⊆RnA\subseteq\mathbb R^nA⊆Rn is semianalytic if near every point it is a finite union of finite intersections of sets {fij=0, gij>0}\{f_{ij}=0,\ g_{ij}>0\}{fij​=0, gij​>0} with fij,gijf_{ij},g_{ij}fij​,gij​ real-analytic. It is subanalytic if near every point it is the projection of a bounded semianalytic subset of Rn×Rm\mathbb R^n\times\mathbb R^mRn×Rm. A function is subanalytic when its graph {(x,λ):f(x)=λ}\{(x,\lambda): f(x)=\lambda\}{(x,λ):f(x)=λ} is. Semialgebraic functions (norms, polynomials, indicators of polyhedra) are subanalytic.

The Fréchet subdifferential ∂^f(x)\hat\partial f(x)∂^f(x) is the set of x∗x^*x∗ with lim inf⁡y→x, y≠x(f(y)−f(x)−⟨x∗,y−x⟩)/∥y−x∥≥0\liminf_{y\to x,\,y\ne x}\big(f(y)-f(x)-\langle x^*,y-x\rangle\big)/\|y-x\|\ge 0liminfy→x,y=x​(f(y)−f(x)−⟨x∗,y−x⟩)/∥y−x∥≥0, for x∈dom⁡fx\in\operatorname{dom} fx∈domf, and is empty otherwise. The limiting subdifferential ∂f(x)\partial f(x)∂f(x) is the set of limits of xk∗∈∂^f(xk)x_k^*\in\hat\partial f(x_k)xk∗​∈∂^f(xk​) with (xk,f(xk))→(x,f(x))(x_k,f(x_k))\to(x,f(x))(xk​,f(xk​))→(x,f(x)). The nonsmooth slope is mf(x)=inf⁡{∥x∗∥:x∗∈∂f(x)}m_f(x)=\inf\{\|x^*\|:x^*\in\partial f(x)\}mf​(x)=inf{∥x∗∥:x∗∈∂f(x)}, equal to +∞+\infty+∞ when ∂f(x)=∅\partial f(x)=\emptyset∂f(x)=∅, and crit⁡f={x:0∈∂f(x)}\operatorname{crit} f=\{x: 0\in\partial f(x)\}critf={x:0∈∂f(x)} is the set of critical points. For lower semicontinuous convex fff, ∂f\partial f∂f is the subdifferential of convex analysis and crit⁡f\operatorname{crit} fcritf is the set of minimizers. Write min⁡f\min fminf for the minimum value and dS(x)d_S(x)dS​(x) for the distance from xxx to S=crit⁡fS=\operatorname{crit} fS=critf. The epigraphical sum g(x)=inf⁡u{f(u)+12∥x−u∥2}g(x)=\inf_u\{f(u)+\tfrac12\|x-u\|^2\}g(x)=infu​{f(u)+21​∥x−u∥2} is the Moreau envelope of fff.

Ratios follow the paper's conventions 00=10^0=100=1 and ∞/∞=0/0=0\infty/\infty=0/0=0∞/∞=0/0=0.

Formalization targets

Goal: Theorem 3.3

Let fff be lower semicontinuous, convex and subanalytic with crit⁡f≠∅\operatorname{crit} f\ne\emptysetcritf=∅. For every bounded set KKK there is θ∈[0,1)\theta\in[0,1)θ∈[0,1) such that

∣f−min⁡f∣θmfis bounded on K.\frac{|f-\min f|^{\theta}}{m_f}\quad\text{is bounded on }K.mf​∣f−minf∣θ​is bounded on K.

The exponent may depend on KKK; neither θ\thetaθ nor the bound is fixed.

Milestones

  1. Eq. (5): ∂f=∂^f=\partial f=\hat\partial f=∂f=∂^f= the convex subdifferential, for lsc convex fff.
  2. Section 3.2: crit⁡f\operatorname{crit} fcritf is closed, convex and equal to the set of minimizers.
  3. Inequality (16): ∣f(x)−min⁡f∣≤∥x∗∥ dS(x)|f(x)-\min f|\le\|x^*\|\,d_S(x)∣f(x)−minf∣≤∥x∗∥dS​(x) for all x∗∈∂f(x)x^*\in\partial f(x)x∗∈∂f(x).
  4. Remark 3.6: ∣f−min⁡f∣/mf|f-\min f|/m_f∣f−minf∣/mf​ is bounded around every critical point, without subanalyticity.
  5. Proposition 2.9: the epigraphical sum ggg is C1C^1C1 and subanalytic when inf⁡f∈R\inf f\in\mathbb Rinff∈R.
  6. Properties (a)–(c): ggg is finite and C1C^1C1, g≤fg\le fg≤f, crit⁡g=crit⁡f\operatorname{crit} g=\operatorname{crit} fcritg=critf, inf⁡g=inf⁡f\inf g=\inf finfg=inff.
  7. Proposition 2.13(ii): crit⁡f\operatorname{crit} fcritf is subanalytic for subanalytic fff that is relatively bounded on its domain.
  8. Section 2.1: the distance to a subanalytic set is subanalytic.
  9. The Łojasiewicz factorization lemma on compact sets (recalled from Bierstone–Milman).
  10. Inequality (15): dS(x)≤c−1/r∣f(x)−min⁡f∣1/rd_S(x)\le c^{-1/r}|f(x)-\min f|^{1/r}dS​(x)≤c−1/r∣f(x)−minf∣1/r on KKK, with r>1r>1r>1, c>0c>0c>0.
  11. Remark 3.5: the growth condition ∣f−min⁡f∣≥c dS r|f-\min f|\ge c\,d_S^{\,r}∣f−minf∣≥cdSr​ on a compact KKK alone yields a Łojasiewicz inequality at critical points interior to KKK.

Significance

Theorem 3.3 gives, for convex subanalytic functions, a Łojasiewicz inequality that is uniform on bounded sets rather than local at one critical point, and it needs neither continuity of fff on its domain nor a closed domain. Remark 3.4 of the paper exhibits a convex function covered by Theorem 3.3 but not by the continuous-case Theorem 3.1. Through inequality (20) of Section 4, it yields finite length and convergence rates for the subgradient flow x˙∈−∂f(x)\dot x\in-\partial f(x)x˙∈−∂f(x) of such functions. The intermediate inequality (15) is a Hölderian error bound, dS≤C∣f−min⁡f∣1/rd_S\le C|f-\min f|^{1/r}dS​≤C∣f−minf∣1/r, of the kind that drives linear and sublinear rate analyses of first-order methods.

The result is proved in the paper; no machine-checked version of it, or of the nonsmooth Łojasiewicz inequality in any form, is known. Formalizing it would add to the library: subanalytic sets and functions, the limiting subdifferential of convex functions and its agreement with the classical one, the Moreau envelope with its critical points and infimum, and the passage from a growth condition to a Łojasiewicz inequality. Remarks 3.5 and 3.6 isolate parts that need no subanalytic geometry at all.

Difficulty

The convex-analysis steps (inequality (16), properties of the Moreau envelope) are classical. The obstacle is subanalytic geometry. The natural first idea, applying the Łojasiewicz factorization lemma directly to f−min⁡ff-\min ff−minf and dSd_SdS​, fails: fff is neither continuous nor finite, and its domain need not be subanalytic even when fff is convex and subanalytic (Example 2.5 of the paper). The milestones route through the Moreau envelope, which is continuous and finite, but subanalyticity is not preserved by infima over unbounded sets, so the subanalyticity of the envelope (Proposition 2.9) needs a localization argument. The subanalyticity of crit⁡g\operatorname{crit} gcritg and of dSd_SdS​ rests on the stability theory of subanalytic sets (Gabrielov's complement theorem, the projection theorem for globally subanalytic sets), none of which exists in Mathlib.

Formalization scope

The space is EuclideanSpace ℝ (Fin n). Functions take values in EReal; "lower semicontinuous, convex, somewhere finite and never −∞-\infty−∞" is the published definition MoreauProx.Characterization.GammaZero, whose convexity is convexity of the epigraph. The Fréchet and limiting subdifferentials are the published NonconvexSplitting.Shared.IsRegularSubgrad and LimitingSubdiff; the convex subdifferential subgrad appears only in Eq. (5), which proves the agreement and is never assumed. Semianalytic and subanalytic sets are defined from scratch for any finite-dimensional real normed space, so that one definition serves Rn\mathbb R^nRn and its products; global subanalyticity is not defined. min⁡f\min fminf is written inf⁡yf(y)\inf_y f(y)infy​f(y) in EReal and converted to a real number only where it is finite. The bounded ratio (14) is encoded as "∣f(x)−min⁡f∣θ≤C∥x∗∥|f(x)-\min f|^{\theta}\le C\|x^*\|∣f(x)−minf∣θ≤C∥x∗∥ for every x∈Kx\in Kx∈K and every x∗∈∂f(x)x^*\in\partial f(x)x∗∈∂f(x)", with real powers (Real.rpow, 00=10^0=100=1). Inequalities (15) and (17) are imposed only where f(x)<+∞f(x)<+\inftyf(x)<+∞, since Lean sends +∞+\infty+∞ to 000 under toReal.

Trivializing encodings are ruled out: the goal is stated with the limiting subdifferential rather than an assumed convex subdifferential, the slope is never computed in ℝ≥0∞ where 0⋅∞=00\cdot\infty=00⋅∞=0 would make the ratio vacuous, and θ\thetaθ remains existential in [0,1)[0,1)[0,1) with the quantifier order "for every KKK there is θ\thetaθ", so that θ=0\theta=0θ=0 is excluded at critical points in KKK by 00=10^0=100=1.

A complete development needs a working theory of subanalytic sets (stability under finite unions, complements, closure, projections of bounded sets, the factorization lemma), the Moreau envelope of a convex function on Rn\mathbb R^nRn and its C1C^1C1 property, and the convex-analytic description of the limiting subdifferential. The subanalytic-geometry layer and the Moreau-envelope facts are reusable well beyond this mission; contributions to either, or proofs of the convex-only milestones (Eq. (5), (16), Remarks 3.5–3.6), are welcome independently.

Selected references

  • J. Bolte, A. Daniilidis, A. Lewis, The Łojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems, SIAM J. Optim. 17(4) (2007) 1205–1223. https://doi.org/10.1137/050644641
  • E. Bierstone, P. D. Milman, Semianalytic and subanalytic sets, Publ. Math. IHÉS 67 (1988) 5–42. https://doi.org/10.1007/BF02699126
  • R. T. Rockafellar, R. J.-B. Wets, Variational Analysis, Springer, 1998. https://doi.org/10.1007/978-3-642-02431-3
  • S. Łojasiewicz, Une propriété topologique des sous-ensembles analytiques réels, Les Équations aux Dérivées Partielles, Éditions du CNRS, Paris, 1963, 87–89.
  • H. Attouch, J. Bolte, B. F. Svaiter, Convergence of descent methods for semi-algebraic and tame problems, Math. Program. 137 (2013) 91–129. https://doi.org/10.1007/s10107-011-0484-9
  • J. Bolte, S. Sabach, M. Teboulle, Proximal alternating linearized minimization for nonconvex and nonsmooth problems, Math. Program. 146 (2014) 459–494. https://doi.org/10.1007/s10107-013-0701-9
18 thms2 active usersReviewed
Operations ResearchProbabilityStochastic Systems+1·Captain: mikedeng1

Approximation Algorithms for Stochastic Inventory Control Models 2: The Triple-Balancing Policy Costs at Most Three Times the Optimum for Stochastic Lot-SizingResearch Paper

Motivation

Periodic-review inventory control with a fixed ordering cost is one of the oldest problems in operations research. A firm reviews its stock at the beginning of each of TTT periods, decides whether to place an order, pays a fixed cost KKK for every order it places, and pays holding costs on leftover stock and penalties on unmet (backlogged) demand. When demand is random and correlated across periods, and the firm's forecast evolves as information arrives, the optimal policy solves a dynamic program over the whole information state. That program is intractable in general, and in practice firms use heuristics with no performance guarantee.

Levi, Pál, Roundy and Shmoys (Math. Oper. Res. 32(2), 2007) gave policies with worst-case guarantees for these models, using a "marginal cost accounting" scheme that charges each unit's holding cost to the period in which it was ordered. For the model with fixed ordering costs, the stochastic lot-sizing problem, they assume that the demand of each period is known at the beginning of that period (make-to-order systems, or settings where the short-term forecast is accurate), while demand further ahead stays random and arbitrarily correlated. Under this assumption they define the triple-balancing policy and prove it costs at most three times the optimum in expectation.

Timeline:

  • Scarf (1960) proved that (s,S)(s,S)(s,S) policies are optimal for independent demands with fixed costs; with correlated demand the optimal policy is a state-dependent (st(ft),St(ft))(s_t(f_t), S_t(f_t))(st​(ft​),St​(ft​)) rule that is hard to compute.
  • Levi, Pál, Roundy and Shmoys (2007) gave the dual-balancing 2-approximation for the model without fixed costs (§4) and the triple-balancing 3-approximation for the stochastic lot-sizing problem (§6, Theorem 6.1), both for arbitrarily correlated demand.

Setting

There are periods t=1,…,Tt=1,\dots,Tt=1,…,T on a probability space (Ω,F,μ)(\Omega,\mathcal F,\mu)(Ω,F,μ) with a filtration (Ft)(\mathcal F_t)(Ft​): Ft\mathcal F_tFt​ is the information available at the beginning of period ttt. The data are a fixed ordering cost K≥0K\ge0K≥0, per-unit holding costs ht≥0h_t\ge0ht​≥0, per-unit backlogging penalties pt≥0p_t\ge0pt​≥0, an initial inventory level x1∈Rx_1\in\mathbb Rx1​∈R, and nonnegative demands DtD_tDt​. The per-unit ordering cost is zero, the lead time is zero and there is no discounting. The defining assumption is that DtD_tDt​ is Ft\mathcal F_tFt​-measurable: the demand of a period is known when the period begins. For every period sss there is a conditional joint distribution IsI_sIs​ of the demands given Fs\mathcal F_sFs​, under which every conditional mean E[Dt∣fs]E[D_t\mid f_s]E[Dt​∣fs​] is finite.

A feasible policy is an order process Q=(Qt)Q=(Q_t)Q=(Qt​) with Qt≥0Q_t\ge0Qt​≥0 and QtQ_tQt​ determined by Ft\mathcal F_tFt​. Its inventory levels are xt=x1+∑j<t(Qj−Dj)x_t=x_1+\sum_{j<t}(Q_j-D_j)xt​=x1​+∑j<t​(Qj​−Dj​) before ordering and yt=xt+Qty_t=x_t+Q_tyt​=xt​+Qt​ after ordering, and its cost is

C(Q)=∑t=1T(K 1(Qt>0)+ht(yt−Dt)++pt(Dt−yt)+).\mathcal C(Q)=\sum_{t=1}^T\Bigl(K\,\mathbb 1(Q_t>0)+h_t(y_t-D_t)^++p_t(D_t-y_t)^+\Bigr).C(Q)=t=1∑T​(K1(Qt​>0)+ht​(yt​−Dt​)++pt​(Dt​−yt​)+).

The triple-balancing policy TB uses two rules. Let s∗s^*s∗ be the last period before sss in which TB ordered (s∗=0s^*=0s∗=0 if none). Rule 1: TB orders in period sss if and only if, without an order in sss, the accumulated backlogging cost over (s∗,s](s^*,s](s∗,s] would exceed KKK. Rule 2: when it orders in s<Ts<Ts<T, it orders

qsB=max⁡{q≥0: E[HsB(q)∣fs]≤K},HsB(q)=∑j=sThj(q−(D[s,j]−xs)+)+,q_s^B=\max\{q\ge0:\ E[H_s^B(q)\mid f_s]\le K\},\qquad H_s^B(q)=\sum_{j=s}^T h_j\bigl(q-(D_{[s,j]}-x_s)^+\bigr)^+,qsB​=max{q≥0: E[HsB​(q)∣fs​]≤K},HsB​(q)=j=s∑T​hj​(q−(D[s,j]​−xs​)+)+,

the largest quantity whose expected marginal holding cost over [s,T][s,T][s,T] is at most KKK. When it orders in period TTT, it orders exactly enough to clear the backorders and meet DTD_TDT​. Let NNN be the number of orders TB places.

Formalization targets

Goal: Theorem 6.1

For every instance, the triple-balancing policy TB and every feasible policy PPP satisfy

E[C(TB)]≤3 E[C(P)].E[\mathcal C(TB)]\le 3\,E[\mathcal C(P)].E[C(TB)]≤3E[C(P)].

The constant 3 is the paper's. The statement leaves the demand law, the information structure and the cost data unrestricted beyond the standing assumptions above.

Milestones

  1. §6.1, Rule 2 observation. In a period where TB orders, Ds≤ysTBD_s\le y_s^{TB}Ds​≤ysTB​: no backorders remain at the end of the period.
  2. Lemma 6.1. K⋅E[N]≤E[C(P)]K\cdot E[N]\le E[\mathcal C(P)]K⋅E[N]≤E[C(P)] for every feasible PPP.
  3. Lemma 6.2. E[C(TB)]≤E[C(P)]+2K⋅E[N]E[\mathcal C(TB)]\le E[\mathcal C(P)]+2K\cdot E[N]E[C(TB)]≤E[C(P)]+2K⋅E[N] for every feasible PPP.

Two non-milestone theorems show that the setting is not empty. A conditional demand law exists whenever demands are integrable, and a triple-balancing policy exists when hT>0h_T>0hT​>0.

Significance

The theorem gives a policy that can be computed online and comes with a worst-case expected-cost guarantee that does not depend on the demand distribution, the horizon or the cost data. In this setting the optimal policy is not computable in general, and the previously used heuristics have no such bound. The two lemmas separate a lower bound on every policy, in terms of TB's own number of orders, from an upper bound on TB's cost. The authors' subsequent work extends the balancing template to capacitated and multi-echelon models (§7 of the paper).

The result is proved in the paper. As far as we know, no machine-checked version exists of this theorem, of the balancing argument, or of a stochastic inventory model with correlated demand and evolving information. A formalization would check the argument, which is terse in places: the printed proof of Lemma 6.2 indexes its final sum loosely and must handle the event N=0N=0N=0. It would also produce reusable infrastructure for policies adapted to a filtration, for regular conditional distributions of future demand, and for cost accounting over random intervals between orders.

Difficulty

The costs of TB and of an arbitrary policy cannot be compared period by period, because the two policies order at different, random times that depend on the evolving information. Any comparison has to be made over intervals whose endpoints are stopping times determined by TB, conditioned on the information at their start. At such a time the other policy may hold more or less stock than TB, and the bound must hold in both cases. Bounding each policy's cost on its own does not work: the guarantee rests on a coupling between when TB orders and what every other policy must pay over the same random stretch of time. The formal side adds a second difficulty. Rule 2 is defined through a conditional expectation viewed as a function of the order quantity, so it needs a regular conditional distribution and a measurable selection of the maximizer.

Formalization scope

  • Periods are natural numbers 1,…,T1,\dots,T1,…,T, demands and orders are real-valued, and data at indices outside 1,…,T1,\dots,T1,…,T are unused.
  • Information is a MeasureTheory.Filtration ℕ. A policy is feasible when it is nonnegative and adapted, and "DtD_tDt​ known at the start of period ttt" means DtD_tDt​ is Ft\mathcal F_tFt​-measurable.
  • The conditional distributions IsI_sIs​ are model data: Markov kernels to demand paths that are Fs\mathcal F_sFs​-measurable regular conditional distributions of the demand path. At every outcome they make DsD_sDs​ deterministic, demands nonnegative and the conditional means E[Dt∣fs]E[D_t\mid f_s]E[Dt​∣fs​] finite.
  • Expected costs, E[N]E[N]E[N] and the conditional expectation in Rule 2 are lower Lebesgue integrals in [0,∞][0,\infty][0,∞]. Lemma 6.2 is stated additively, E[C(TB)]≤E[C(P)]+2K E[N]E[\mathcal C(TB)]\le E[\mathcal C(P)]+2K\,E[N]E[C(TB)]≤E[C(P)]+2KE[N], which is the paper's inequality whenever the expectations are finite.
  • The comparison policy is an arbitrary feasible policy, not an optimal one. The paper's proofs use only feasibility, and this form implies the paper's whenever an optimum exists, without any existence hypothesis.
  • TB is the predicate "feasible and satisfies Rules 1 and 2 at every period and outcome". The rules determine the policy uniquely. Rule 1 uses a strict "exceeds KKK", and the period-TTT order is DT−xTD_T-x_TDT​−xT​.

Several trivializing formalizations are ruled out. Junk conditional expectations cannot make Rule 2 hold for every qqq, because it uses kernel integrals in [0,∞][0,\infty][0,∞]. Infinite expected costs cannot be read as 000. The policy class is not empty, because a separate theorem gives existence under hT>0h_T>0hT​>0 (without some positive holding cost on [s,T][s,T][s,T] the maximum in Rule 2 does not exist).

Contributions welcome: proofs of the existence theorems (measurable selection of qsBq_s^BqsB​, versions of regular conditional distributions), the stopping-time decomposition of the cost over TB's order intervals, and Lemmas 6.1 and 6.2.

Selected references

  • R. Levi, M. Pál, R. O. Roundy, D. B. Shmoys, Approximation Algorithms for Stochastic Inventory Control Models, Mathematics of Operations Research 32(2):284–302, 2007. https://doi.org/10.1287/moor.1060.0205
  • H. Scarf, The Optimality of (S, s) Policies in the Dynamic Inventory Problem, in Mathematical Methods in the Social Sciences, Stanford University Press, 1960.
7 thms2 active usersReviewed
🏆Completed
Markov ChainOperations ResearchProbability+1·Captain: mikedeng1

Optimization of Multiclass Queueing Networks: Polyhedral and Nonlinear Characterizations of Achievable Performance I: Quadratic Potential Functions Bound Mean Response Times in Open NetworksResearch Paper

Motivation

Scheduling in a multiclass queueing network asks which waiting job a server should work on next when jobs of several types share stations and revisit them along fixed routes. Such networks model semiconductor wafer fabs, job shops and communication switches. Optimal policies are rarely computable: the state space is countably infinite, and even deciding properties of optimal policies is hard (Papadimitriou and Tsitsiklis 1999). A practical substitute is the achievable region approach: describe, by constraints that every policy must satisfy, a set containing all performance vectors any policy can achieve, then optimize a linear cost over that set to get a lower bound on the optimal cost.

Bertsimas, Paschalidis and Tsitsiklis (MIT Sloan working paper 1992; Ann. Appl. Probab. 1994) gave a general method for producing such constraints for open networks, by computing the steady-state drift of quadratic potential functions. This mission formalizes their first-order bounds (Section 4).

Timeline:

  • 1980–1988: Coffman and Mitrani, then Federgruen and Groenevelt — the achievable performance vectors of a single-station multiclass queue form a polytope described by conservation laws.
  • Early 1990s: Kumar (reference [Kuma] of the paper), using a potential-function argument he attributes to Meyn, derives a single lower bound on the mean number in system for re-entrant lines with deterministic routing (described on p. 16 of the paper).
  • 1992–1994: Bertsimas, Paschalidis and Tsitsiklis — parametric families of linear bounds for general open networks with Markovian routing (Theorem 4.1), and the nonparametric polyhedron (Theorems 4.2–4.4), shown to be at least as tight.

Setting

A network has NNN single-server stations and RRR job classes. Class rrr is served at station σ(r)\sigma(r)σ(r), and CiC_iCi​ is the set of classes served at station iii. Class-rrr jobs arrive from outside as a Poisson stream of rate λ0r\lambda_{0r}λ0r​, service times are exponential with rate μr\mu_rμr​, and after service a class-rrr job becomes a class-sss job with probability prsp_{rs}prs​ or leaves with probability pr0=1−∑sprsp_{r0}=1-\sum_s p_{rs}pr0​=1−∑s​prs​. The traffic equations

λr=λ0r+∑r′λr′pr′r(15)\lambda_r=\lambda_{0r}+\sum_{r'}\lambda_{r'}p_{r'r}\qquad(15)λr​=λ0r​+r′∑​λr′​pr′r​(15)

have a unique solution λ\lambdaλ (the network is open), and ∑r∈Ciλr/μr<1\sum_{r\in C_i}\lambda_r/\mu_r<1∑r∈Ci​​λr​/μr​<1 at every station.

The state n⃗=(n1,…,nR)\vec n=(n_1,\dots,n_R)n=(n1​,…,nR​) counts the jobs of each class. A Markovian policy decides from the current state which classes are in service, at most one per station and only classes with jobs present; idling is allowed. Write BrB_rBr​ for the event that station σ(r)\sigma(r)σ(r) serves class rrr, and B0iB_{0i}B0i​ for the event that station iii is idle. Under such a policy n⃗(t)\vec n(t)n(t) is a continuous-time Markov chain. Assumption A requires that it has a unique invariant distribution π\piπ and that Eπ[nr2]<∞E_\pi[n_r^2]<\inftyEπ​[nr2​]<∞ for all rrr. Let nˉr=Eπ[nr]\bar n_r=E_\pi[n_r]nˉr​=Eπ​[nr​], which equals λrxr\lambda_rx_rλr​xr​ with xrx_rxr​ the mean response time of class rrr (Little's law), and define

Irr′=Eπ[1{Br}nr′],Nir′=Eπ[1{B0i}nr′].I_{rr'}=E_\pi[1\{B_r\}n_{r'}],\qquad N_{ir'}=E_\pi[1\{B_{0i}\}n_{r'}].Irr′​=Eπ​[1{Br​}nr′​],Nir′​=Eπ​[1{B0i​}nr′​].

For a set SSS of classes, f-parameters are reals f(r)≥0f(r)\ge 0f(r)≥0 for r∈Sr\in Sr∈S such that μr[∑r′∈Sprr′(f(r)−f(r′))+∑r′∉Sprr′f(r)]\mu_r\big[\sum_{r'\in S}p_{rr'}(f(r)-f(r'))+\sum_{r'\notin S}p_{rr'}f(r)\big]μr​[∑r′∈S​prr′​(f(r)−f(r′))+∑r′∈/S​prr′​f(r)] is nonnegative and the same for all r∈Ci∩Sr\in C_i\cap Sr∈Ci​∩S; that common value is fif_ifi​, and fi=0f_i=0fi​=0 when Ci∩S=∅C_i\cap S=\emptysetCi​∩S=∅ (restriction (17)). The sums over r′∉Sr'\notin Sr′∈/S include the exit r′=0r'=0r′=0.

Formalization targets

Goal: Theorem 4.1

For every policy satisfying Assumption A, every SSS and every f-parameters satisfying (17),

∑r∈Sλrf(r)xr ≥ N′(S)D′(S),\sum_{r\in S}\lambda_rf(r)x_r\ \ge\ \frac{N'(S)}{D'(S)},r∈S∑​λr​f(r)xr​ ≥ D′(S)N′(S)​,

where

N′(S)=∑r∈Sλ0rf2(r)+∑r∉Sλr∑r′∈Sprr′f2(r′)+∑r∈Sλr[∑r′∈Sprr′(f(r)−f(r′))2+∑r′∉Sprr′f2(r)],N'(S)=\sum_{r\in S}\lambda_{0r}f^2(r)+\sum_{r\notin S}\lambda_r\sum_{r'\in S}p_{rr'}f^2(r')+\sum_{r\in S}\lambda_r\Big[\sum_{r'\in S}p_{rr'}(f(r)-f(r'))^2+\sum_{r'\notin S}p_{rr'}f^2(r)\Big],N′(S)=r∈S∑​λ0r​f2(r)+r∈/S∑​λr​r′∈S∑​prr′​f2(r′)+r∈S∑​λr​[r′∈S∑​prr′​(f(r)−f(r′))2+r′∈/S∑​prr′​f2(r)], D′(S)=2[∑i=1Nfi−∑r∈Sλ0rf(r)].D'(S)=2\Big[\sum_{i=1}^Nf_i-\sum_{r\in S}\lambda_{0r}f(r)\Big].D′(S)=2[i=1∑N​fi​−r∈S∑​λ0r​f(r)].

The formal goal is the product form N′(S)≤D′(S)∑r∈Sf(r)nˉrN'(S)\le D'(S)\sum_{r\in S}f(r)\bar n_rN′(S)≤D′(S)∑r∈S​f(r)nˉr​.

Milestones

  1. The utilization identity Eπ[1{Br}]=λr/μrE_\pi[1\{B_r\}]=\lambda_r/\mu_rEπ​[1{Br​}]=λr​/μr​ (pp. 16 and 19).
  2. Theorem 4.2: the linear equalities (24), (25) between nˉr\bar n_rnˉr​ and Irr′I_{rr'}Irr′​.
  3. Theorem 4.3: ∑r∈CiIrr′+Nir′=nˉr′\sum_{r\in C_i}I_{rr'}+N_{ir'}=\bar n_{r'}∑r∈Ci​​Irr′​+Nir′​=nˉr′​ (28).
  4. Theorem 4.4: any nonnegative (x,I,N)(x,I,N)(x,I,N) satisfying (24), (25), (28), with nˉr=λrxr\bar n_r=\lambda_rx_rnˉr​=λr​xr​ in those equalities, satisfies every inequality of Theorem 4.1. This statement is deterministic.

Significance

Theorem 4.1 gives, for each choice of SSS and fff, a linear inequality on mean response times valid for all admissible policies. Minimizing a linear holding cost ∑rcrxr\sum_r c_rx_r∑r​cr​xr​ subject to these inequalities is a linear program whose value bounds the optimal scheduling cost from below; the paper reports numerical values of such bounds in its Section 9. Theorems 4.2–4.4 show that a polynomial-size polyhedron in the variables (nˉ,I,N)(\bar n,I,N)(nˉ,I,N) implies all of these inequalities at once, so the parametric search over fff is unnecessary.

The results are proved in the paper. As far as is known, none of them has a machine-checked proof. Formalizing them requires a Lean treatment of invariant distributions of controlled countable-state Markov chains with unbounded test functions, which is currently absent from Mathlib, and then the algebra of the drift identities. The definitions here (network data, Markovian sequencing policies, the generator, Assumption A) are the substrate that the paper's later results on routing, closed networks and higher-order bounds would reuse.

Difficulty

Every statement except Theorem 4.4 rests on taking expectations of the generator applied to unbounded functions (nrn_rnr​, nrnr′n_rn_{r'}nr​nr′​) under the invariant distribution. The invariance condition is stated only for indicators of single states; extending ∑nπ(n)(Gg)(n)=0\sum_n\pi(n)(\mathcal Gg)(n)=0∑n​π(n)(Gg)(n)=0 to quadratic ggg needs an interchange of summations justified by the second-moment condition of Assumption A. The utilization identity additionally needs uniqueness of the traffic solution to identify μrEπ[1{Br}]\mu_rE_\pi[1\{B_r\}]μr​Eπ​[1{Br​}] with λr\lambda_rλr​. Theorem 4.1 then needs the sign bookkeeping that turns an identity into an inequality: the terms dropped are nonnegative only because f≥0f\ge0f≥0 on SSS, fi≥0f_i\ge0fi​≥0 and at most one class per station is in service.

Formalization scope

Classes are Fin R, stations Fin N, states Fin R → ℕ, all rates and probabilities real. A policy is a Bool-valued function of the state with the two admissibility constraints; work conservation is not assumed. Invariance is global balance of the generator on the countable state space; expectations are tsums. The uniformized chain and the epochs τk\tau_kτk​ of the paper are not built: the paper notes that its expectations at τk\tau_kτk​ are expectations under the invariant distribution of n⃗(t)\vec n(t)n(t).

Conventions fixed in Lean:

  • λrxr\lambda_rx_rλr​xr​ appears only as the mean number in system nˉr\bar n_rnˉr​ (Little's law, used by the paper on pp. 11 and 20); response times are not formalized.
  • Sums over r′∉Sr'\notin Sr′∈/S include the exit r′=0r'=0r′=0 (p. 15).
  • f-parameters are nonnegative on SSS (p. 9).
  • The network is open: (15) has a unique solution, and λ\lambdaλ is an input constrained by (15), never defined from the policy.
  • (18) is stated multiplied by D′(S)D'(S)D′(S), which avoids Lean's x/0=0x/0=0x/0=0 and is (18) whenever D′(S)>0D'(S)>0D′(S)>0.

A quotient-form statement of (18) would be trivially true when D′(S)=0D'(S)=0D′(S)=0, and defining λr\lambda_rλr​ as μrEπ[1{Br}]\mu_rE_\pi[1\{B_r\}]μr​Eπ​[1{Br​}] would make the utilization identity hold by definition; both are excluded.

Welcome contributions: a general lemma extending global balance to test functions of polynomial growth under moment conditions; proofs of the drift identities; the deterministic Theorem 4.4.

Selected references

  • D. Bertsimas, I. Ch. Paschalidis, J. N. Tsitsiklis, Optimization of Multiclass Queueing Networks: Polyhedral and Nonlinear Characterizations of Achievable Performance, MIT Sloan WP #3509-92-MSA, 1992; Ann. Appl. Probab. 4(1), 1994. https://doi.org/10.1214/aoap/1177005200
  • C. H. Papadimitriou, J. N. Tsitsiklis, The complexity of optimal queuing network control, Math. Oper. Res. 24(2), 1999. https://doi.org/10.1287/moor.24.2.293
8 thms2 active usersReviewed
AnalysisDynamical SystemsOperations Research+1·Captain: mikedeng1

The Łojasiewicz Inequality for Nonsmooth Subanalytic Functions with Applications to Subgradient Dynamical Systems III: Bounded Subgradient Trajectories Converge with Łojasiewicz RatesResearch Paper

Motivation

Many optimization algorithms are discretizations of a continuous-time descent: the gradient flow x˙=−∇f(x)\dot x=-\nabla f(x)x˙=−∇f(x) for smooth objectives, and its nonsmooth analogue, the subgradient dynamical system, for objectives with kinks or constraints. A basic question about such a flow is whether a bounded trajectory actually converges, rather than merely accumulating on a continuum of critical points, and how fast. For real-analytic fff this was settled by Łojasiewicz through his gradient inequality, which forces bounded gradient trajectories to have finite length. Without some such structure the answer is negative: there are smooth functions whose bounded gradient trajectories spiral forever around a circle of critical points.

Bolte, Daniilidis and Lewis (SIAM J. Optim. 17 (2007) 1205–1223) extended the Łojasiewicz inequality to nonsmooth subanalytic functions, possibly taking the value +∞+\infty+∞, by replacing ∥∇f∥\|\nabla f\|∥∇f∥ with the least norm of a limiting subgradient. Section 4 of the paper turns this inequality into convergence results for subgradient trajectories of convex and lower-C2C^2C2 functions. This mission formalizes that section. The same "Łojasiewicz argument" later became the Kurdyka–Łojasiewicz framework behind convergence proofs for proximal, alternating and splitting algorithms (Attouch–Bolte 2009; Bolte–Sabach–Teboulle 2014).

Timeline. Łojasiewicz (1963, 1984) proved the gradient inequality for real-analytic functions and finite length of bounded analytic gradient trajectories. Kurdyka (Ann. Inst. Fourier 1998) extended it to C1C^1C1 functions definable in an o-minimal structure. Kurdyka, Mostowski and Parusiński (2000) proved Thom's gradient conjecture for analytic functions. Bolte, Daniilidis and Lewis (2007) gave the nonsmooth subanalytic version and the trajectory results formalized here.

Setting

Let f:Rn→R∪{+∞}f:\mathbb R^n\to\mathbb R\cup\{+\infty\}f:Rn→R∪{+∞} with domain dom⁡f={x:f(x)<+∞}\operatorname{dom} f=\{x: f(x)<+\infty\}domf={x:f(x)<+∞}. The Fréchet subdifferential ∂^f(x)\hat\partial f(x)∂^f(x) is the set of x∗x^*x∗ with lim inf⁡y→x, y≠x(f(y)−f(x)−⟨x∗,y−x⟩)/∥y−x∥≥0\liminf_{y\to x,\,y\neq x}\big(f(y)-f(x)-\langle x^*,y-x\rangle\big)/\|y-x\|\ge0liminfy→x,y=x​(f(y)−f(x)−⟨x∗,y−x⟩)/∥y−x∥≥0. The limiting subdifferential ∂f(x)\partial f(x)∂f(x) is the set of limits of xk∗∈∂^f(xk)x_k^*\in\hat\partial f(x_k)xk∗​∈∂^f(xk​) with xk→xx_k\to xxk​→x and f(xk)→f(x)f(x_k)\to f(x)f(xk​)→f(x). The nonsmooth slope is mf(x)=inf⁡{∥x∗∥:x∗∈∂f(x)}m_f(x)=\inf\{\|x^*\|: x^*\in\partial f(x)\}mf​(x)=inf{∥x∗∥:x∗∈∂f(x)}, equal to +∞+\infty+∞ when ∂f(x)=∅\partial f(x)=\emptyset∂f(x)=∅, and crit⁡f={x:0∈∂f(x)}\operatorname{crit} f=\{x: 0\in\partial f(x)\}critf={x:0∈∂f(x)} is the set of critical points.

The standing assumptions of Section 4 are:

  • (H1)(\mathcal H1)(H1) fff is either lower semicontinuous and convex, or lower-C2C^2C2 with dom⁡f=Rn\operatorname{dom} f=\mathbb R^ndomf=Rn. Lower-C2C^2C2 means that near each point f=max⁡s∈SF(⋅,s)f=\max_{s\in S}F(\cdot,s)f=maxs∈S​F(⋅,s) for a compact space SSS and a jointly continuous FFF with jointly continuous first and second xxx-derivatives.
  • (H2)(\mathcal H2)(H2) fff is somewhere finite and bounded from below.
  • (H3)(\mathcal H3)(H3) fff is subanalytic: its graph is locally the projection of a bounded set defined by finitely many real-analytic equalities and strict inequalities.

A trajectory of the subgradient system (G)(\mathcal G)(G) is an absolutely continuous curve x:[0,T)→Rnx:[0,T)\to\mathbb R^nx:[0,T)→Rn, T∈(0,+∞]T\in(0,+\infty]T∈(0,+∞], with x˙(t)+∂f(x(t))∋0\dot x(t)+\partial f(x(t))\ni0x˙(t)+∂f(x(t))∋0 for almost every ttt and ∂f(x(t))≠∅\partial f(x(t))\neq\emptyset∂f(x(t))=∅ for every ttt. It is maximal if it admits no extension to a longer interval. The Łojasiewicz inequality holds around aaa with exponent θ\thetaθ if ∣f−f(a)∣θ/mf|f-f(a)|^\theta/m_f∣f−f(a)∣θ/mf​ is bounded near aaa, with 00=10^0=100=1 and ∞/∞=0/0=0\infty/\infty=0/0=0∞/∞=0/0=0. A Łojasiewicz exponent at a∈dom⁡fa\in\operatorname{dom} fa∈domf is any such θ∈[0,1)\theta\in[0,1)θ∈[0,1).

Formalization targets

Goal: Theorem 4.7

Under (H1)(\mathcal H1)(H1)–(H3)(\mathcal H3)(H3), every bounded maximal trajectory xxx is defined on [0,+∞)[0,+\infty)[0,+∞) and converges to a critical point aaa. For every Łojasiewicz exponent θ\thetaθ at aaa there are k,k′>0k,k'>0k,k′>0 and t0≥0t_0\ge0t0​≥0 such that for t≥t0t\ge t_0t≥t0​

∥x(t)−a∥≤{k (t+1)−1−θ2θ−1,θ∈(12,1),k e−k′t,θ=12,\|x(t)-a\|\le \begin{cases} k\,(t+1)^{-\frac{1-\theta}{2\theta-1}}, & \theta\in(\tfrac12,1),\\[2pt] k\,e^{-k't}, & \theta=\tfrac12,\end{cases}∥x(t)−a∥≤{k(t+1)−2θ−11−θ​,ke−k′t,​θ∈(21​,1),θ=21​,​

and for θ∈[0,12)\theta\in[0,\tfrac12)θ∈[0,21​), x(t)=ax(t)=ax(t)=a for all large ttt. The constants are existential, so the goal survives any later sharpening of them.

Milestones

  1. Corollary 4.1(i): for almost every ttt, ddtf(x(t))=⟨x˙(t),x∗⟩\frac{d}{dt}f(x(t))=\langle\dot x(t),x^*\rangledtd​f(x(t))=⟨x˙(t),x∗⟩ for every x∗∈∂f(x(t))x^*\in\partial f(x(t))x∗∈∂f(x(t)).
  2. Corollary 4.1(iii): every trajectory extends to a maximal one on [0,+∞)[0,+\infty)[0,+∞) with x˙∈L2\dot x\in L^2x˙∈L2.
  3. Corollary 4.2: ∥x˙(t)∥=mf(x(t))\|\dot x(t)\|=m_f(x(t))∥x˙(t)∥=mf​(x(t)) and ddtf(x(t))=−mf(x(t))2\frac{d}{dt}f(x(t))=-m_f(x(t))^2dtd​f(x(t))=−mf​(x(t))2 almost everywhere.
  4. Inequality (20): the Łojasiewicz inequality holds around every point of dom⁡∂f\operatorname{dom}\partial fdom∂f.
  5. Theorem 4.5: bounded maximal trajectories have finite length ∫0∞∥x˙∥<∞\int_0^\infty\|\dot x\|<\infty∫0∞​∥x˙∥<∞ and converge to a critical point.
  6. The tail bound ∫t∞∥x˙∥≤c1−θ(f(x(t))−f(a))1−θ\int_t^\infty\|\dot x\|\le\frac{c}{1-\theta}(f(x(t))-f(a))^{1-\theta}∫t∞​∥x˙∥≤1−θc​(f(x(t))−f(a))1−θ.
  7. Inequality (27): ∫t∞∥x˙∥≤c1/θ1−θ∥x˙(t)∥(1−θ)/θ\int_t^\infty\|\dot x\|\le\frac{c^{1/\theta}}{1-\theta}\|\dot x(t)\|^{(1-\theta)/\theta}∫t∞​∥x˙∥≤1−θc1/θ​∥x˙(t)∥(1−θ)/θ for almost every large ttt.

Significance

Theorem 4.5 says that for convex or lower-C2C^2C2 subanalytic objectives, including constrained problems through indicator functions of subanalytic sets, the subgradient flow never oscillates indefinitely: bounded trajectories converge to one critical point. Theorem 4.7 adds rates that depend only on the Łojasiewicz exponent at the limit: exponential at θ=12\theta=\tfrac12θ=21​, polynomial above it, finite time below it. These are continuous-time templates for the convergence analyses of proximal and splitting methods under the Kurdyka–Łojasiewicz property.

On the formalization side, the results are proved on paper but, to our knowledge, not formalized in any proof assistant. A development would provide reusable infrastructure: a Lean notion of a trajectory of a differential inclusion on [0,T)[0,T)[0,T), a chain rule for f∘xf\circ xf∘x along absolutely continuous curves, and a comparison lemma for the differential inequality σ˙≤−Lσα\dot\sigma\le-L\sigma^\alphaσ˙≤−Lσα. The analysis of Section 4 uses subanalyticity only through inequality (20), so Theorems 4.5 and 4.7 can be attacked with (20) as an imported milestone, independently of the subanalytic geometry.

Difficulty

Compactness gives cluster points of a bounded trajectory, and the decrease of fff gives convergence of f(x(t))f(x(t))f(x(t)). Neither gives convergence of x(t)x(t)x(t). The usual first idea, that ∫0∞∥x˙∥2<∞\int_0^\infty\|\dot x\|^2<\infty∫0∞​∥x˙∥2<∞ forces convergence, fails, since square-integrable speed allows infinite length. The difficulty is to control ∫∥x˙∥\int\|\dot x\|∫∥x˙∥ rather than ∫∥x˙∥2\int\|\dot x\|^2∫∥x˙∥2. That needs a lower bound on the slope in terms of the function gap near the cluster point, which is exactly what (20) supplies, plus a trapping argument showing the tail of the trajectory stays in the ball where (20) holds. In the nonsmooth setting the chain rule itself is nontrivial: f∘xf\circ xf∘x is differentiable almost everywhere with derivative ⟨x˙,x∗⟩\langle\dot x,x^*\rangle⟨x˙,x∗⟩ for every x∗∈∂f(x(t))x^*\in\partial f(x(t))x∗∈∂f(x(t)), which relies on ∂f=∂^f\partial f=\hat\partial f∂f=∂^f for convex and lower-C2C^2C2 functions. Global existence on [0,+∞)[0,+\infty)[0,+∞) (Corollary 4.1(iii)) must also be established before any asymptotic statement makes sense.

Formalization scope

Space Rn\mathbb R^nRn is EuclideanSpace ℝ (Fin n). Functions take values in EReal; "bounded from below" by a real number excludes −∞-\infty−∞. The limiting subdifferential is the published NonconvexSplitting.Shared.LimitingSubdiff, and convexity is the published MoreauProx.Characterization.EConvex (convex epigraph). The slope mfm_fmf​ is valued in [0,+∞][0,+\infty][0,+∞]. The Łojasiewicz inequality is encoded as ∣f(y)−f(a)∣θ≤C∥v∥|f(y)-f(a)|^\theta\le C\|v\|∣f(y)−f(a)∣θ≤C∥v∥ for yyy near aaa and every v∈∂f(y)v\in\partial f(y)v∈∂f(y), which is the bounded ratio under the paper's conventions. Times are real numbers and T∈[0,+∞]T\in[0,+\infty]T∈[0,+∞]. Curves are functions R→Rn\mathbb R\to\mathbb R^nR→Rn whose values outside [0,T)[0,T)[0,T) are irrelevant. "Absolutely continuous on [0,T)[0,T)[0,T)" means absolutely continuous on every compact [0,b]⊆[0,T)[0,b]\subseteq[0,T)[0,b]⊆[0,T). Velocities appear only "for almost every ttt". Lengths are lower Lebesgue integrals ∫−∥x˙∥\int^-\|\dot x\|∫−∥x˙∥ in [0,+∞][0,+\infty][0,+∞], never Bochner integrals (which would vanish for a non-integrable speed).

Two trivializations are ruled out. Maximality is a hypothesis and T=+∞T=+\inftyT=+∞ is a conclusion: assuming T=+∞T=+\inftyT=+∞ would narrow the theorem, and dropping maximality would make it false. Rates are claimed for every Łojasiewicz exponent at the limit, not for one chosen exponent. Corollary 4.1(iii) is stated as "defined on R+\mathbb R_+R+​ with x^˙∈L2\dot{\hat x}\in L^2x^˙∈L2", because the printed x^∈W1,2(R+)\hat x\in W^{1,2}(\mathbb R_+)x^∈W1,2(R+​) would fail for any trajectory with nonzero limit.

Needed infrastructure: absolutely continuous curves and their a.e. derivatives (Mathlib's AbsolutelyContinuousOnInterval), chain rules for convex and lower-C2C^2C2 functions, existence and uniqueness for monotone differential inclusions (Brézis), and an ODE comparison principle. Contributions to any of these are reusable well beyond this mission.

Selected references

  • J. Bolte, A. Daniilidis, A. Lewis, The Łojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems, SIAM J. Optim. 17(4) (2007) 1205–1223. https://doi.org/10.1137/050644641
  • H. Brézis, Opérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert, North-Holland, 1973.
  • J.-P. Aubin, A. Cellina, Differential Inclusions, Springer, 1984. https://doi.org/10.1007/978-3-642-69512-4
  • R. T. Rockafellar, R. J.-B. Wets, Variational Analysis, Springer, 1998. https://doi.org/10.1007/978-3-642-02431-3
  • K. Kurdyka, On gradients of functions definable in o-minimal structures, Ann. Inst. Fourier 48 (1998) 769–783. https://doi.org/10.5802/aif.1638
  • H. Attouch, J. Bolte, On the convergence of the proximal algorithm for nonsmooth functions involving analytic features, Math. Program. 116 (2009) 5–16. https://doi.org/10.1007/s10107-007-0133-5
17 thms1 active userReviewed
🏆Completed
Algorithmic Game TheoryOperations Research·Captain: mikedeng1

The Price of Stability for Network Design with Fair Cost Allocation III: Weighted Games in Which Each Edge Serves at Most Two Players Have a Potential and a Nash EquilibriumResearch Paper

Motivation

In a network design game each of kkk players must connect its own terminals in a shared graph, and the cost of every edge that is bought is split among the players who use it. Anshelevich, Dasgupta, Kleinberg, Tardos, Wexler and Roughgarden (SIAM J. Comput. 2008) studied the fair (Shapley) split, in which the xex_exe​ users of an edge each pay ce/xec_e/x_ece​/xe​. That game is a congestion game in the sense of Rosenthal (Networks 1973), so it has an exact potential function and pure Nash equilibria always exist.

When players carry different amounts of traffic, the natural rule is to split an edge's cost in proportion to weight: a player of weight wiw_iwi​ on an edge whose users have total weight WeW_eWe​ pays (wi/We) ce(w_i/W_e)\,c_e(wi​/We​)ce​. The paper notes that this rule is analogous to weighted generalizations of the Shapley value (Monderer and Samet, Variations of the Shapley Value, Handbook of Game Theory III, 2002). The weighted model leaves Rosenthal's framework: the share depends on which players use an edge, not only on how many, and Chen and Roughgarden (SPAA 2006) showed that weighted games with three or more players need not have a pure Nash equilibrium at all. Section 6 of the paper identifies structural conditions under which equilibria do exist. This mission covers the first of them.

Timeline. Rosenthal (1973): every congestion game has a pure Nash equilibrium, via an exact potential. Monderer and Shapley (GEB 1996): potential and weighted potential games, and the equivalence of exact potential games with congestion games. Anshelevich et al. (FOCS 2004, journal 2008): Theorem 6.1, existence when every resource is shared by at most two players, and Theorem 6.3, existence when all players share a source and a sink. Chen and Roughgarden (2006): weighted network design games with three or more players may have no pure equilibrium.

Setting

A weighted cost-sharing game GGG consists of a finite set of players, a finite ground set EEE of edges (resources), and for each player iii:

  • a finite family Σi\Sigma_iΣi​ of feasible strategies, each a subset of EEE;
  • a weight wi≥1w_i \ge 1wi​≥1;

together with a fixed edge cost ce≥0c_e \ge 0ce​≥0 for every e∈Ee \in Ee∈E. A profile S=(Si)iS = (S_i)_iS=(Si​)i​ picks Si∈ΣiS_i \in \Sigma_iSi​∈Σi​ for every player. For an edge eee, WeW_eWe​ is the total weight of the players with e∈Sie \in S_ie∈Si​, and player iii's payment is

Ci(S)=∑e∈SiwiWe ce.C_i(S) = \sum_{e \in S_i} \frac{w_i}{W_e}\, c_e .Ci​(S)=e∈Si​∑​We​wi​​ce​.

A profile is a pure Nash equilibrium when no player iii has a T∈ΣiT \in \Sigma_iT∈Σi​ with Ci(S−i,T)<Ci(S)C_i(S_{-i}, T) < C_i(S)Ci​(S−i​,T)<Ci​(S), where (S−i,T)(S_{-i}, T)(S−i​,T) is the profile in which iii plays TTT and everyone else keeps their strategy.

The strategy space of player iii is the set of edges that occur in at least one strategy of Σi\Sigma_iΣi​. The hypothesis of Theorem 6.1 is that every edge lies in the strategy spaces of at most two players: no edge can ever be shared by three players, whatever they choose.

The network design game is the instance in which EEE is the edge set of a graph and Σi\Sigma_iΣi​ is the set of edge sets of paths connecting player iii's source sis_isi​ to its sink tit_iti​.

The paper's proof uses an explicit function Φ(S)=∑eΦe(S)\Phi(S) = \sum_e \Phi_e(S)Φ(S)=∑e​Φe​(S) with Φe(S)=0\Phi_e(S) = 0Φe​(S)=0 when eee is unused, cewic_e w_ice​wi​ when iii alone uses eee, and ceθijc_e\theta_{ij}ce​θij​ when iii and jjj both use it, where θij=wi+wj−wiwj/(wi+wj)\theta_{ij} = w_i + w_j - w_i w_j/(w_i + w_j)θij​=wi​+wj​−wi​wj​/(wi​+wj​). It is part of the definitions of this mission.

Formalization targets

Goal: Theorem 6.1

If every edge lies in the strategy spaces of at most two players, there is a weighted potential: a real function Φ\PhiΦ on profiles with

Φ(S−i,T)−Φ(S)=wi (Ci(S−i,T)−Ci(S))for every profile S, player i, T∈Σi,\Phi(S_{-i}, T) - \Phi(S) = w_i\,\bigl(C_i(S_{-i}, T) - C_i(S)\bigr) \quad\text{for every profile } S,\ \text{player } i,\ T \in \Sigma_i ,Φ(S−i​,T)−Φ(S)=wi​(Ci​(S−i​,T)−Ci​(S))for every profile S, player i, T∈Σi​,

and, if every Σi\Sigma_iΣi​ is nonempty, a pure Nash equilibrium exists. The goal asserts the existence of such a Φ\PhiΦ rather than fixing the paper's formula, so it remains valid for any other weighted potential.

Milestones

  1. Joining a shared edge (proof of Theorem 6.1): when iii joins an edge already used by exactly one other player jjj, Φe\Phi_eΦe​ rises by cewi2/(wi+wj)c_e w_i^2/(w_i + w_j)ce​wi2​/(wi​+wj​), which is wiw_iwi​ times iii's new share of eee.
  2. The identity for the explicit potential: the displayed identity holds for the paper's Φ\PhiΦ.
  3. From a weighted potential to an equilibrium: in any weighted game with positive weights and nonempty strategy sets, a function satisfying the identity forces a pure Nash equilibrium to exist.

An extra item states Corollary 6.2: every two-player weighted game with nonempty strategy sets has a pure Nash equilibrium.

Significance

The result. Theorem 6.1 is one of the two existence results the paper proves for weighted cost sharing, a game that in general has no pure equilibrium. It shows that the obstruction found by Chen and Roughgarden needs resources shared by three or more players: whenever sharing is limited to pairs, the game is a weighted potential game, so improving moves cannot cycle and equilibria exist. Corollary 6.2 makes the two-player case unconditional, and the paper notes that the same potential gives a (weak) bound on the price of stability.

Formalizing it. The result is proved in the paper; to our knowledge no machine-checked version exists. The mission produces a reusable Lean model of weight-proportional cost sharing (shared in form with the companion mission on single-source single-sink weighted games), an explicit weighted potential, and the general step from a weighted potential to a pure equilibrium, which applies to any finite game with positive weights.

Difficulty

The obvious approach, reusing Rosenthal's potential from the unweighted game, fails: the paper observes that in a weighted game improving moves can increase it. A player's share of an edge depends on the weights of the specific co-users, so no function of the edge loads alone can track all players' costs. The identity must therefore hold for every unilateral move, including moves that leave some edges and join others at the same time, and for every pair of possible co-users of an edge. The statement fails without the at-most-two hypothesis, so any argument has to use it in an essential way. The existence step needs the identity on all profiles reachable by feasible deviations, not only along a single path of moves.

Formalization scope

Lean namespace PriceOfStability.WeightedPotential. Players form a Fintype ι and edges a Fintype E; a game is a structure with strategies : ι → Finset (Finset E), weight : ι → ℝ and edgeCost : E → ℝ. Standing assumptions wᵢ ≥ 1 and c_e ≥ 0 are the predicate IsStandard. Profiles are functions ι → Finset E with the feasibility predicate IsProfile; every deviation is to a feasible strategy, via Function.update. Nash equilibria are pure and in cost form. The strategy-space hypothesis is a bound on the number of players whose strategy space (the union of their strategies) contains each edge — not a bound on the users in one profile, which would be a different statement. Φ_e is computed from the current users of e; its value with three or more users is a placeholder that never arises under the hypothesis. Strategies are arbitrary subsets of the ground set, as the paper's remark after the proof allows, so the network game is a special case.

The goal is not satisfiable trivially: the function Φ must satisfy the weighted identity for every feasible unilateral deviation from every profile, and an exact (unweighted) potential is not what is asserted. Nonempty strategy sets are added explicitly for the existence part, since without a profile there is no equilibrium.

Needed infrastructure: finite sums over filtered Finsets, the improvement-path argument over the finite set of profiles. The improvement-path lemma (milestone 3) is reusable for any weighted potential game. Contributions of proofs for any milestone are welcome.

Selected references

  • E. Anshelevich, A. Dasgupta, J. Kleinberg, É. Tardos, T. Wexler, T. Roughgarden, The Price of Stability for Network Design with Fair Cost Allocation, SIAM Journal on Computing 38(4):1602–1623, 2008. https://doi.org/10.1137/070680096
  • R. W. Rosenthal, The network equilibrium problem in integers, Networks 3:53–59, 1973. https://doi.org/10.1002/net.3230030104
  • D. Monderer, L. S. Shapley, Potential games, Games and Economic Behavior 14:124–143, 1996. https://doi.org/10.1006/game.1996.0044
  • H.-L. Chen, T. Roughgarden, Network design with weighted players, Proc. 18th ACM SPAA, 28–37, 2006. https://doi.org/10.1145/1148109.1148114
5 thms2 active usersReviewed
🏆Completed
Operations ResearchOptimizationTheoretical Computer Science·Captain: mikedeng1

Optimal Sequencing of a Single Machine Subject to Precedence Constraints: Repeatedly Placing Last a Least-Cost Eligible Job Yields a Minmax Optimal SequenceResearch Paper

Motivation

Single-machine sequencing is the base case of deterministic scheduling theory. Many multi-machine and shop problems are analysed by reduction to it, and many bounds and approximation algorithms for harder models use it as a subroutine. A central objective class is the bottleneck or minmax objective. Each job carries a nondecreasing cost of its completion time, and the schedule is judged by its worst job. Maximum lateness, maximum tardiness and maximum weighted tardiness are all special cases.

Before 1973 the minmax problem was solved without precedence constraints. Jackson (1955) showed that ordering by due date minimizes maximum lateness. Moore (1968, Management Science 15(1)) gave a procedure for general nondecreasing deferral costs, and Lawler and Moore (1969) gave a related method. In Lawler, Optimal Sequencing of a Single Machine Subject to Precedence Constraints, Management Science 19(5), 1973, Lawler showed that arbitrary precedence constraints can be added at no loss of efficiency. Jobs are chosen from last to first, by a single comparison of costs at a known time. The resulting O(n2)O(n^2)O(n2) procedure is the standard algorithm for the problem written 1 ∣ prec ∣ fmax⁡1\,|\,\mathrm{prec}\,|\,f_{\max}1∣prec∣fmax​ in the classification of Graham, Lawler, Lenstra and Rinnooy Kan (1979). It is one of the first polynomial-time results for precedence-constrained scheduling that every survey of the field cites.

Setting

A finite, nonempty set JJJ of jobs is processed on a single machine, one job at a time and without interruption. Each job jjj has a processing time aj≥0a_j \ge 0aj​≥0 and a cost function cj:R→Rc_j : \mathbb{R} \to \mathbb{R}cj​:R→R that is monotone nondecreasing. The value cj(t)c_j(t)cj​(t) is the cost incurred when jjj is completed at time ttt.

The precedence constraints are an arbitrary relation ≺\prec≺ on jobs: i≺ji \prec ji≺j means that job iii is required to precede job jjj. A sequence π=(π1,…,πn)\pi = (\pi_1, \dots, \pi_n)π=(π1​,…,πn​) lists every job of JJJ once. It observes the precedence constraints if πq≺πp\pi_q \prec \pi_pπq​≺πp​ never holds for positions p<qp < qp<q. The machine starts at time 000 with no idle time, so the completion time of πm\pi_mπm​ is Cπm(π)=aπ1+⋯+aπmC_{\pi_m}(\pi) = a_{\pi_1} + \dots + a_{\pi_m}Cπm​​(π)=aπ1​​+⋯+aπm​​. The maximum incurred cost of π\piπ is

fmax⁡(π)=max⁡j∈Jcj(Cj(π)),f_{\max}(\pi) = \max_{j \in J} c_j\bigl(C_j(\pi)\bigr),fmax​(π)=j∈Jmax​cj​(Cj​(π)),

and a feasible π\piπ is minmax optimal if fmax⁡(π)≤fmax⁡(π′)f_{\max}(\pi) \le f_{\max}(\pi')fmax​(π)≤fmax​(π′) for every feasible π′\pi'π′.

For a set PPP of jobs, S(P)S(P)S(P) is the set of jobs of PPP that are not required to precede any other job of PPP, and TP=∑j∈PajT_P = \sum_{j \in P} a_jTP​=∑j∈P​aj​. Lawler's rule builds a sequence from the last position to the first. With PPP the jobs not yet placed, it chooses k∈S(P)k \in S(P)k∈S(P) with ck(TP)=min⁡j∈S(P)cj(TP)c_k(T_P) = \min_{j \in S(P)} c_j(T_P)ck​(TP​)=minj∈S(P)​cj​(TP​), places kkk in the latest open position and removes it from PPP. Ties are broken arbitrarily. In Lean the objects are IsFeasible, lastEligible (SSS), IsMinmaxOptimal and IsLawlerSequence, in namespace LawlerPrec.MinMax. They are built on the published MooreLateJobs.Shared.completionTime and MooreLateJobs.MaxDeferral.maxCost.

Formalization targets

Goal: the rule is optimal

Every sequence π\piπ that Lawler's rule can produce, under any tie-breaking, observes the precedence constraints and satisfies

fmax⁡(π)  ≤  fmax⁡(π′)for every sequence π′ of J observing the precedence constraints.f_{\max}(\pi) \;\le\; f_{\max}(\pi') \qquad \text{for every sequence } \pi' \text{ of } J \text{ observing the precedence constraints.}fmax​(π)≤fmax​(π′)for every sequence π′ of J observing the precedence constraints.

This is the statement of §3 (p. 545), "An efficient algorithm for finding a minmax optimal sequence follows immediately from the theorem above". It contains no constants.

Milestones

  1. §2 proof, third paragraph. Moving a job of S(J)S(J)S(J) to the end of a feasible sequence keeps it feasible.
  2. §2 proof, fourth paragraph, first sentence. After that move, no job other than kkk completes later, and kkk completes at T=∑j∈JajT = \sum_{j \in J} a_jT=∑j∈J​aj​.
  3. §2 proof, fourth paragraph. If ck(T)≤ck′(T)c_k(T) \le c_{k'}(T)ck​(T)≤ck′​(T), where k′k'k′ is the last job of the feasible sequence, the move does not raise fmax⁡f_{\max}fmax​.
  4. THEOREM (§2), p. 544. If some feasible sequence exists and k∈S(J)k \in S(J)k∈S(J) minimizes cj(T)c_j(T)cj​(T) over S(J)S(J)S(J), then some minmax optimal sequence has kkk last.
  5. §3, the reduction. A minmax optimal sequence of J∖{k}J \setminus \{k\}J∖{k}, followed by kkk, is minmax optimal for JJJ.
  6. §3, the procedure never stalls. If a feasible sequence exists, the rule produces a complete sequence. This shows the goal is not vacuous.

Significance

The result shows that 1 ∣ prec ∣ fmax⁡1\,|\,\mathrm{prec}\,|\,f_{\max}1∣prec∣fmax​ is solvable in polynomial time for every family of nondecreasing costs. The ordering of an optimal sequence depends on the costs only through their values at the nnn partial sums TPT_PTP​ along the way. The deadline problem is a corollary (§5): sequencing from last to first by latest deadline among the currently available jobs avoids tardiness whenever any sequence does. The last-to-first scheme is reused in later backward rules for fmax⁡f_{\max}fmax​ objectives. A formal statement of the rule, its feasibility and its optimality makes these extensions available for formal reuse.

The result is classical and its proof is short. No machine-checked proof of it is known to be in Mathlib. The work this mission asks for is a formal proof of the known exchange argument and of the induction that turns the Theorem into the algorithm's correctness. The induction needs the reduced problem's sets S(P)S(P)S(P) and times TPT_PTP​ to be the correct ones at each stage, which the definitions fix.

Difficulty

The exchange argument of §2 is elementary. The difficulty lies in stating the algorithm faithfully and carrying the induction. At each stage the eligible set S(P)S(P)S(P) and the time TPT_PTP​ must be recomputed on the remaining jobs, with constraints into already placed jobs ignored. The induction must also show that the rule's sequence is feasible, which is a conclusion and not an assumption.

A first attempt often proves only the Theorem, that some optimal sequence has kkk last. That statement says nothing about a sequence built entirely by the rule, because an optimal sequence of JJJ with kkk last need not restrict to an optimal sequence of J∖{k}J \setminus \{k\}J∖{k}. Optimality of the rule's whole sequence is the target, and milestone 5 isolates the corresponding step of the page.

Formalization scope

  • Jobs form a type ι with decidable equality, and the job set is J : Finset ι.
  • Processing times are a : ι → ℝ, costs are c : ι → ℝ → ℝ, and the precedence constraints are prec : ι → ι → Prop.
  • A sequence is a duplicate-free list whose elements are exactly J. Positions are 0-based, and completion times are prefix sums (MooreLateJobs.Shared.completionAt).
  • The relation prec is arbitrary: it is not assumed transitive, irreflexive or acyclic. A cycle among distinct jobs leaves no feasible sequence. A self-loop constrains nothing, both in feasibility and in SSS (the "others" of the page exclude the job itself).

The standing assumptions of §1 appear as hypotheses wherever they are used: monotone nondecreasing cjc_jcj​ for j∈Jj \in Jj∈J, and JJJ nonempty where the maximum is taken. Two hypotheses are added relative to the page and disclosed in each statement. Processing times are non-negative (aj≥0a_j \ge 0aj​≥0), since they are durations and the exchange argument fails without them. The Theorem also assumes the existence of a feasible sequence, which its conclusion presupposes.

The rule is the property IsLawlerSequence of a finished sequence. At each position mmm, the job there lies in SSS of the jobs in positions 0..m0..m0..m and minimizes the cost at their total processing time. Every tie-break is covered. The rule is not a deterministic function, and it is not an arbitrary choice function. Feasibility of the rule's output is part of the goal's conclusion, so the goal cannot be obtained by assuming it. A statement that compares the rule only with some sequence, or that asserts only that an optimal sequence exists, is weaker and is ruled out by the goal's form. Milestone 6 shows the goal's hypotheses are satisfiable whenever a feasible sequence exists.

The n2n^2n2 operation count of §4, the first-to-last rule of §5 and the deadline corollaries of §5 are not part of this mission. A development needs only finite lists and finsets from Mathlib. Lemmas about moving an element to the end of a duplicate-free list, and about prefix sums under that move, are reusable for other exchange arguments in single-machine scheduling.

Selected references

  • E. L. Lawler, Optimal Sequencing of a Single Machine Subject to Precedence Constraints, Management Science 19(5):544–546, 1973. https://doi.org/10.1287/mnsc.19.5.544
  • J. M. Moore, An n Job, One Machine Sequencing Algorithm for Minimizing the Number of Late Jobs, Management Science 15(1):102–109, 1968. https://doi.org/10.1287/mnsc.15.1.102
  • E. L. Lawler and J. M. Moore, A Functional Equation and its Application to Resource Allocation and Sequencing Problems, Management Science 16(1):77–84, 1969. https://doi.org/10.1287/mnsc.16.1.77
  • J. R. Jackson, Scheduling a Production Line to Minimize Maximum Tardiness, Research Report 43, Management Science Research Project, UCLA, 1955.
  • R. L. Graham, E. L. Lawler, J. K. Lenstra and A. H. G. Rinnooy Kan, Optimization and Approximation in Deterministic Sequencing and Scheduling: a Survey, Annals of Discrete Mathematics 5:287–326, 1979. https://doi.org/10.1016/S0167-5060(08)70356-X
13 thms2 active usersReviewed
🏆Completed
Algorithmic Game TheoryOperations Research·Captain: mikedeng1

The Price of Stability for Network Design with Fair Cost Allocation I: The Price of Stability of Fair Cost Sharing Is at Most H(k), and This Is TightResearch Paper

Motivation

Many networks are built and paid for by the users they serve: multicast trees, virtual overlays, shared subnetworks of the Internet. A protocol proposes a design and a rule for splitting its cost, and each participant is free to accept the proposal or defect to a cheaper alternative. The designer therefore cannot impose the global optimum; the best it can do is propose the cheapest outcome that no participant wants to leave, a Nash equilibrium. The ratio between the cost of the best equilibrium and the optimal cost, the price of stability, measures the loss caused by requiring stability. It contrasts with the price of anarchy, which compares the worst equilibrium with the optimum and suits settings with no coordinating protocol at all.

Anshelevich, Dasgupta, Kleinberg, Tardos, Wexler and Roughgarden (SIAM J. Comput. 2008; preliminary version FOCS 2004) studied this question for the most common cost-sharing rule, the Shapley (equal-split) rule, under which every user of an edge pays the same share of its cost. Their first result, the subject of this mission, is that the price of stability is at most the harmonic number H(k)H(k)H(k), where kkk is the number of players, and that this bound is attained in the limit.

Setting

There are kkk players and a finite set EEE of edges. Each player iii has a family Σi\Sigma_iΣi​ of feasible strategies, each a set of edges; in the network setting Σi\Sigma_iΣi​ consists of the edge sets connecting player iii's terminals in a directed graph. Each edge eee has a nonnegative cost cec_ece​. In a strategy vector S=(S1,…,Sk)S=(S_1,\dots,S_k)S=(S1​,…,Sk​), Si∈ΣiS_i\in\Sigma_iSi​∈Σi​, let xex_exe​ be the number of players whose strategy contains eee. Under Shapley cost sharing player iii pays

Ci(S)=∑e∈Sicexe.C_i(S)=\sum_{e\in S_i}\frac{c_e}{x_e}.Ci​(S)=e∈Si​∑​xe​ce​​.

The cost of the designed network is

cost⁡(S)=∑e∈⋃iSice,\operatorname{cost}(S)=\sum_{e\in\bigcup_i S_i}c_e ,cost(S)=e∈⋃i​Si​∑​ce​,

and the payments add up to exactly this amount. The profile SSS is a (pure) Nash equilibrium if no player iii has a strategy Si′∈ΣiS_i'\in\Sigma_iSi′​∈Σi​ with Ci(S−i,Si′)<Ci(S)C_i(S_{-i},S_i')<C_i(S)Ci​(S−i​,Si′​)<Ci​(S). Finally,

H(k)=1+12+⋯+1k.H(k)=1+\tfrac12+\dots+\tfrac1k .H(k)=1+21​+⋯+k1​.

The game is a congestion game: the per-user cost of an edge, fe(x)=ce/xf_e(x)=c_e/xfe​(x)=ce​/x, depends only on the edge and its number of users. Rosenthal's potential

Φ(S)=∑e∈E∑x=1xefe(x)\Phi(S)=\sum_{e\in E}\sum_{x=1}^{x_e}f_e(x)Φ(S)=e∈E∑​x=1∑xe​​fe​(x)

changes by exactly the deviating player's change in cost when a single player changes strategy. The same objects are used with load-dependent edge costs ce(x)c_e(x)ce​(x), in which case fe(x)=ce(x)/xf_e(x)=c_e(x)/xfe​(x)=ce​(x)/x.

Formalization targets

Goal: Theorem 2.1 and its tightness

For every game with ce≥0c_e\ge0ce​≥0 in which each player has a feasible strategy there is a Nash equilibrium SSS with

cost⁡(S)≤H(k)⋅cost⁡(P)for every profile P,\operatorname{cost}(S)\le H(k)\cdot\operatorname{cost}(P)\quad\text{for every profile }P,cost(S)≤H(k)⋅cost(P)for every profile P,

and for every k≥1k\ge1k≥1, ε>0\varepsilon>0ε>0 the instance of Fig. 1.1 (player iii has its own path of cost 1/i1/i1/i, and all players can share a path of cost 1+ε1+\varepsilon1+ε) has a Nash equilibrium, every Nash equilibrium of it costs H(k)H(k)H(k), and some profile costs 1+ε1+\varepsilon1+ε. The ratio H(k)/(1+ε)H(k)/(1+\varepsilon)H(k)/(1+ε) tends to H(k)H(k)H(k) as ε→0\varepsilon\to0ε→0.

Milestones

  1. Budget balance: ∑iCi(S)=cost⁡(S)\sum_iC_i(S)=\operatorname{cost}(S)∑i​Ci​(S)=cost(S) for every SSS (Sect. 1).
  2. Rosenthal's potential is exact (Theorem 2.1, proof, (2.1)).
  3. From every profile some Nash equilibrium of no larger potential is reached (Theorem 2.1, proof).
  4. Theorem 3.1: if cost⁡(S)≤A Φ(S)\operatorname{cost}(S)\le A\,\Phi(S)cost(S)≤AΦ(S) and Φ(S)≤Bcost⁡(S)\Phi(S)\le B\operatorname{cost}(S)Φ(S)≤Bcost(S) for all SSS, the price of stability is at most ABABAB.
  5. For nondecreasing concave edge costs ce(x)c_e(x)ce​(x): cost⁡(S)≤Φ(S)≤H(k)cost⁡(S)\operatorname{cost}(S)\le\Phi(S)\le H(k)\operatorname{cost}(S)cost(S)≤Φ(S)≤H(k)cost(S) (Theorem 2.3, proof).
  6. Theorem 2.3: the H(k)H(k)H(k) bound for nondecreasing concave edge costs.
  7. The Fig. 1.1 instance: its Nash equilibrium is unique and costs H(k)H(k)H(k); a profile costs 1+ε1+\varepsilon1+ε.

Significance

The bound shows that requiring stability under the Shapley rule costs at most a logarithmic factor, H(k)=Θ(log⁡k)H(k)=\Theta(\log k)H(k)=Θ(logk), whereas the price of anarchy of the same game is kkk (two parallel edges of costs 111 and kkk already show this). It was among the first price-of-stability results and is the starting point for a line of work on network design games: the undirected case, where the H(k)H(k)H(k) bound is not tight and the correct value remained open for years, weighted players, and other cost-sharing rules. The argument (an exact potential that over- and under-estimates the social cost by bounded factors) is the standard tool for price-of-stability bounds, and Theorem 3.1 isolates it in a reusable form.

The theorem is proved in the paper; to our knowledge no machine-checked proof exists. This mission produces a formal account of Shapley cost-sharing games as congestion games, of Rosenthal's potential and the finite improvement property, of the potential-sandwich argument of Theorem 3.1, and of the matching lower-bound instance, for constant and for nondecreasing concave edge costs.

Difficulty

The obvious approach, bounding the cost of an arbitrary equilibrium, fails: some equilibria cost kkk times the optimum, so any proof must select a particular equilibrium. The selection uses the finiteness of the strategy space together with the exact potential, and the bound needs the inequality Φ≤H(k)⋅cost⁡\Phi\le H(k)\cdot\operatorname{cost}Φ≤H(k)⋅cost, which for concave costs requires the per-user cost ce(x)/xc_e(x)/xce​(x)/x to be nonincreasing. On the lower-bound side, the claim that every equilibrium of Fig. 1.1 costs H(k)H(k)H(k) requires excluding all equilibria in which some players share the common path, for every kkk at once, not only checking that the all-own profile is stable.

Formalization scope

Games are encoded as congestion games over arbitrary finite families of edge sets, reusing the published CongestionPoA.AsymSum.Model (congestion game, loads, player costs, cost-form pure Nash equilibrium, total cost). The paper notes that its proofs do not use the graph structure; the directed-graph game is the instance in which Σi\Sigma_iΣi​ is the family of edge sets connecting player iii's terminals. Players and edges form finite types; kkk is the number of players and H(k)H(k)H(k) is Mathlib's harmonic k cast to R\mathbb RR. The price of stability is stated as the existence of a Nash equilibrium whose cost is at most the constant times the cost of every profile, with no division by the optimum, and with the hypothesis that some profile exists. Concave costs are functions on N\mathbb NN with nonincreasing increments, nondecreasing, with ce(0)≥0c_e(0)\ge0ce​(0)≥0; this last condition is implicit in the paper and needed for ce(x)/xc_e(x)/xce​(x)/x to be nonincreasing. Theorem 3.1 carries the implicit hypothesis A≥0A\ge0A≥0.

A statement that bounds every equilibrium is false, and one that asserts a cheap profile without the Nash condition is trivial; both are excluded, and the tightness part quantifies over every Nash equilibrium of the instance and asserts that one exists.

Needed infrastructure: finite improvement paths in potential games, sum manipulations over loads, and bounds on harmonic sums. The potential lemmas apply to every finite congestion game and are reusable beyond this mission. Proofs of any milestone are welcome independently.

Selected references

  • E. Anshelevich, A. Dasgupta, J. Kleinberg, É. Tardos, T. Wexler, T. Roughgarden, The Price of Stability for Network Design with Fair Cost Allocation, SIAM J. Comput. 38(4):1602–1623, 2008. https://doi.org/10.1137/070680096
  • R. W. Rosenthal, A class of games possessing pure-strategy Nash equilibria, Int. J. Game Theory 2:65–67, 1973. https://doi.org/10.1007/BF01737559
  • D. Monderer, L. S. Shapley, Potential games, Games Econ. Behav. 14:124–143, 1996. https://doi.org/10.1006/game.1996.0044
11 thms3 active usersReviewed
🏆Completed
AnalysisDynamic ProgrammingOperations Research+3·Captain: mikedeng1

Stochastic Optimal Control: The Discrete-Time Case V: Semicontinuous Functions — a Borel-Measurable Minimizing Selector for Lower Semicontinuous CostsTextbook

Motivation

Every step of the dynamic programming algorithm on a general state space does three things: it takes a conditional expectation of the cost-to-go under a transition kernel, it minimizes the resulting function of state and control over the control, and, if a policy is to be produced, it picks a control for each state that attains or nearly attains that minimum. On a finite or countable state space all three are harmless. On an uncountable state space each can destroy the measurability needed to take the next expectation: the infimum over an uncountable family of measurable functions need not be measurable, and a minimizer chosen state by state need not be a measurable function of the state, so it does not define a policy at all.

Section 7.5 of Bertsekas and Shreve, Stochastic Optimal Control: The Discrete-Time Case (1978; Athena Scientific reprint 1996), settles the three operations for semicontinuous costs and continuous kernels. The results are the topological half of the book's measurability theory; the descriptive set theory half (lower semianalytic functions and analytically measurable selectors, §7.6–7.7) is a separate mission in this series. The semicontinuous results are what Propositions 8.6–8.7 and Corollaries 9.17.2–9.17.3 of the book use to obtain Borel-measurable optimal policies for finite-horizon and infinite-horizon models with lower semicontinuous costs and compact control sets.

Timeline. The exact selection theorem for lower semicontinuous functions (Proposition 7.33 below) is credited by the book's notes to Dubins and Savage, How to Gamble If You Must (1965). The Hausdorff metric on closed sets goes back to Hausdorff's Set Theory. Measurable selection in the closed-valued setting was later systematized by Kuratowski and Ryll-Nardzewski (1965), whose theorem gives a different route to results of this kind.

Setting

Throughout, R∗=[−∞,+∞]R^*=[-\infty,+\infty]R∗=[−∞,+∞] is the extended real line. A function f:X→R∗f:X\to R^*f:X→R∗ on a metrizable space XXX is lower semicontinuous if every sublevel set {x∣f(x)≤c}\{x\mid f(x)\le c\}{x∣f(x)≤c}, c∈Rc\in\mathbb Rc∈R, is closed, and upper semicontinuous if every superlevel set {x∣f(x)≥c}\{x\mid f(x)\ge c\}{x∣f(x)≥c} is closed (Definition 7.13). C(X)C(X)C(X) is the space of bounded continuous real-valued functions on XXX.

For a separable metrizable space YYY, P(Y)P(Y)P(Y) is the set of Borel probability measures on YYY with the weak topology (convergence of integrals of functions in C(Y)C(Y)C(Y)). A stochastic kernel q(dy∣x)q(dy\mid x)q(dy∣x) on YYY given XXX is a map x↦q(dy∣x)x\mapsto q(dy\mid x)x↦q(dy∣x) from XXX to P(Y)P(Y)P(Y), and it is continuous if this map is continuous (Definition 7.12). The integral of a Borel-measurable f:Y→R∗f:Y\to R^*f:Y→R∗ is ∫f dp=∫f+dp−∫f−dp\int f\,dp=\int f^+dp-\int f^-dp∫fdp=∫f+dp−∫f−dp with the convention −∞+∞=+∞−∞=+∞-\infty+\infty=+\infty-\infty=+\infty−∞+∞=+∞−∞=+∞ (Eq. (43) of Chapter 7).

For a compact metric space YYY, 2Y2^Y2Y is the collection of closed subsets of YYY with the topology of the Hausdorff metric (Appendix C). For D⊆X×YD\subseteq X\times YD⊆X×Y, the section at xxx is Dx={y∣(x,y)∈D}D_x=\{y\mid (x,y)\in D\}Dx​={y∣(x,y)∈D}, the projection is projX(D)={x∣Dx≠∅}\mathrm{proj}_X(D)=\{x\mid D_x\neq\emptyset\}projX​(D)={x∣Dx​=∅}, and a function φ:projX(D)→Y\varphi:\mathrm{proj}_X(D)\to Yφ:projX​(D)→Y has its graph in DDD if (x,φ(x))∈D(x,\varphi(x))\in D(x,φ(x))∈D for every x∈projX(D)x\in\mathrm{proj}_X(D)x∈projX​(D). "Borel-measurable" refers to the Borel σ-algebras of the topologies in question; on projX(D)\mathrm{proj}_X(D)projX​(D) this is the Borel σ-algebra of the subspace topology.

Formalization targets

Goal: Proposition 7.33

Let XXX be metrizable, YYY compact metrizable, D⊆X×YD\subseteq X\times YD⊆X×Y closed, and f:D→R∗f:D\to R^*f:D→R∗ lower semicontinuous. Put

f∗(x)=min⁡y∈Dxf(x,y),x∈projX(D).f^*(x)=\min_{y\in D_x}f(x,y),\qquad x\in\mathrm{proj}_X(D).f∗(x)=y∈Dx​min​f(x,y),x∈projX​(D).

Then projX(D)\mathrm{proj}_X(D)projX​(D) is closed, f∗f^*f∗ is lower semicontinuous, and there is a Borel-measurable φ:projX(D)→Y\varphi:\mathrm{proj}_X(D)\to Yφ:projX​(D)→Y with graph in DDD and

f(x,φ(x))=f∗(x)∀x∈projX(D).f\bigl(x,\varphi(x)\bigr)=f^*(x)\qquad\forall x\in\mathrm{proj}_X(D).f(x,φ(x))=f∗(x)∀x∈projX​(D).

Milestones

  • Proposition 7.32: for f∗(x)=inf⁡y∈Yf(x,y)f^*(x)=\inf_{y\in Y}f(x,y)f∗(x)=infy∈Y​f(x,y), lower semicontinuity of fff and compactness of YYY give lower semicontinuity of f∗f^*f∗ and attainment; upper semicontinuity of fff gives upper semicontinuity of f∗f^*f∗.
  • Lemma 7.18: there is a Borel-measurable σ:2Y−{∅}→Y\sigma:2^Y-\{\emptyset\}\to Yσ:2Y−{∅}→Y with σ(A)∈A\sigma(A)\in Aσ(A)∈A.
  • Lemma 7.20: for lower semicontinuous fff on a nonempty compact YYY, the argmin map x↦{y∣f(x,y)≤f∗(x)}x\mapsto\{y\mid f(x,y)\le f^*(x)\}x↦{y∣f(x,y)≤f∗(x)} is Borel-measurable into 2Y2^Y2Y.
  • Lemma 7.14: fff is lower semicontinuous and bounded below iff fn↑ff_n\uparrow ffn​↑f for some fn∈C(X)f_n\in C(X)fn​∈C(X) (and dually).
  • Proposition 7.30: x↦∫f(x,y) q(dy∣x)x\mapsto\int f(x,y)\,q(dy\mid x)x↦∫f(x,y)q(dy∣x) is continuous for f∈C(X×Y)f\in C(X\times Y)f∈C(X×Y) and continuous qqq.
  • Proposition 7.31: the same map is lower (upper) semicontinuous and bounded below (above) when fff is.
  • Lemma 7.21: an open G⊆X×YG\subseteq X\times YG⊆X×Y, YYY separable, has open projection and a Borel-measurable selector with graph in GGG.
  • Proposition 7.34: for open DDD and upper semicontinuous fff, projX(D)\mathrm{proj}_X(D)projX​(D) is open, f∗=inf⁡Dxff^*=\inf_{D_x}ff∗=infDx​​f is upper semicontinuous, and for each ε>0\varepsilon>0ε>0 there is a Borel-measurable φε\varphi_\varepsilonφε​ with graph in DDD and
f(x,φε(x))≤{f∗(x)+εif f∗(x)>−∞,−1/εif f∗(x)=−∞.f\bigl(x,\varphi_\varepsilon(x)\bigr)\le\begin{cases}f^*(x)+\varepsilon&\text{if }f^*(x)>-\infty,\\-1/\varepsilon&\text{if }f^*(x)=-\infty.\end{cases}f(x,φε​(x))≤{f∗(x)+ε−1/ε​if f∗(x)>−∞,if f∗(x)=−∞.​

Significance

The results. Propositions 7.31–7.33 are the closure properties that make the dynamic programming recursion stay inside the class of lower semicontinuous functions bounded below: the expectation step preserves the class (7.31), the minimization step preserves it (7.32, 7.33), and the minimization admits a Borel-measurable exact minimizer (7.33). This is why, in semicontinuous models, the optimal cost functions are lower semicontinuous and optimal policies can be taken Borel-measurable and nonrandomized. Proposition 7.34 gives the weaker, ε\varepsilonε-optimal counterpart for upper semicontinuous costs, where the infimum need not be attained.

Formalizing them. All of these results are proved in the book; none is open. As far as is known, none has a machine-checked proof: Mathlib has semicontinuity, the Hausdorff extended metric on closed and on nonempty compact sets, and the weak topology on probability measures, but no theorem combining them into a measurable selection result of this kind. A formal development would supply measurable selectors for semicontinuous minimization in Lean and the Borel-measurability of set-valued maps into the hyperspace of closed sets, both reusable well beyond dynamic programming.

Difficulty

The obvious attempt at the goal is to pick, for each xxx, some minimizer yyy of f(x,⋅)f(x,\cdot)f(x,⋅) over the compact section DxD_xDx​. The minimizer exists by compactness and lower semicontinuity, but the choice is made pointwise and gives no control on measurability: a minimizer chosen by the axiom of choice need not be Borel-measurable. The argmin sets F∗(x)F^*(x)F∗(x) vary with xxx only semicontinuously: they can jump from a single point to a large set, so a continuous selection generally does not exist, and continuity arguments cannot replace measurability. Lemma 7.18 isolates the hardest part: a choice of a point of each nonempty closed set that is measurable as a function of the set itself.

A second difficulty is bookkeeping at infinity. Values ±∞\pm\infty±∞ are allowed throughout, so sublevel sets, minima, integrals and ε\varepsilonε-bounds must all be handled in R∗R^*R∗; the integral in Proposition 7.31 uses the convention ∞−∞=+∞\infty-\infty=+\infty∞−∞=+∞, which is not Mathlib's.

Formalization scope

  • Extended reals. Values are in EReal. The only place where values of opposite infinite sign are combined is the integral, which is the published definition DupacovaWets.Consistency.expect (reused, not restated): ∫f+−∫f−\int f^+-\int f^-∫f+−∫f− with an explicit case returning +∞+\infty+∞ when ∫f+=∞\int f^+=\infty∫f+=∞, exactly the book's convention (42). The ε\varepsilonε-bound of Proposition 7.34 adds a real ε\varepsilonε to a value different from −∞-\infty−∞, which is safe in EReal.
  • Semicontinuity is Mathlib's LowerSemicontinuous/UpperSemicontinuous, equivalent to Definition 7.13 for EReal-valued functions. Lemma 7.13 of the book (the sequential characterization) is Mathlib's lowerSemicontinuous_iff_le_liminf together with first countability of metrizable spaces, and is not restated here.
  • Functions on DDD. Functions "on DDD" are functions on X×YX\times YX×Y with LowerSemicontinuousOn f D (resp. UpperSemicontinuousOn); values off DDD play no role. projX(D)\mathrm{proj}_X(D)projX​(D) is Prod.fst '' D, selectors are functions on that subtype, and its σ-algebra is the Borel σ-algebra of the subspace topology.
  • Hyperspace. 2Y2^Y2Y is Closeds Y, and 2Y−{∅}2^Y-\{\emptyset\}2Y−{∅} for compact YYY is NonemptyCompacts Y, each with the Hausdorff extended metric and the Borel σ-algebra of its topology. This topology agrees with the book's (the exponential topology of Appendix C, independent of the metric).
  • Boundedness. "Bounded below/above" is by a real constant. BddBelow in EReal would be vacuous and is not used.
  • Edge cases. Proposition 7.32(a)'s attainment clause is stated for nonempty YYY, since for Y=∅Y=\emptysetY=∅ the infimum is +∞+\infty+∞ and nothing attains it.
  • Argmin minimum. Lemma 7.20 assumes nonempty YYY because its defining formula uses a minimum; for empty YYY there is no minimizer.
  • Ruling out trivial readings. The graph condition (x,φ(x))∈D(x,\varphi(x))\in D(x,φ(x))∈D is part of every selection statement; without it the goal would follow from the unconstrained case. The selector must be Borel-measurable on projX(D)\mathrm{proj}_X(D)projX​(D) and must attain the minimum exactly, not up to ε\varepsilonε.

A complete development needs the Borel structure of the hyperspace (measurability of maps into Closeds Y from upper semicontinuity in the sense of Kuratowski, Proposition C.4 of the book), the construction of a measurable choice function on NonemptyCompacts Y, and approximation of semicontinuous functions by monotone sequences in C(X)C(X)C(X). Each of these is reusable on its own; proofs of individual milestones by any route are welcome.

Selected references

  • D. P. Bertsekas and S. E. Shreve, Stochastic Optimal Control: The Discrete-Time Case, Academic Press, 1978; Athena Scientific reprint, 1996, Section 7.5 and Appendix C. https://web.mit.edu/dimitrib/www/soc.html
  • L. E. Dubins and L. J. Savage, How to Gamble If You Must: Inequalities for Stochastic Processes, McGraw-Hill, 1965.
  • K. Kuratowski and C. Ryll-Nardzewski, "A general theorem on selectors," Bull. Acad. Polon. Sci. 13 (1965), 397–403.
  • F. Hausdorff, Set Theory, Chelsea, New York, 1957.
10 thms2 active usersReviewed
🏆Completed
Algorithmic Game TheoryOperations Research·Captain: mikedeng1

The Price of Stability for Network Design with Fair Cost Allocation IV: In Weighted Games with a Common Source and Sink, Best-Response Dynamics Converge to a Nash EquilibriumResearch Paper

Motivation

In a network design game each player must connect its terminals in a graph whose edges carry fixed costs, and the cost of an edge is split among the players that use it. Anshelevich, Dasgupta, Kleinberg, Tardos, Wexler and Roughgarden (SIAM J. Comput. 38 (2008)) studied the fair (Shapley) split, in which the users of an edge pay equal shares. That game is a congestion game in the sense of Rosenthal (Int. J. Game Theory 2 (1973)), so it has an exact potential and pure Nash equilibria always exist.

Section 6 of the same paper turns to weighted players: player iii has a weight wi≥1w_i \ge 1wi​≥1 (a traffic volume, a bandwidth demand, a share of ownership) and pays for each edge it uses a share proportional to its weight. The equal-split potential is then lost, and the paper notes that weighted games with three or more players need not have a pure Nash equilibrium at all (Chen and Roughgarden, Network design with weighted players, SPAA 2006). Theorem 6.3 identifies a natural class in which equilibria survive: all players share one source and one sink. For that class it shows more than existence. The simplest decentralized procedure, letting players in turn switch to a cheapest route, always stops, and where it stops is an equilibrium.

Setting

A finite directed multigraph DDD has a finite set EEE of arcs; each arc eee has a tail and a head vertex, and parallel arcs between the same two vertices are allowed. Fix a source sss and a sink ttt. A simple sss–ttt path is a sequence of arcs e1,…,eme_1,\dots,e_me1​,…,em​ (m≥1m\ge1m≥1), each starting where the previous one ends, beginning at sss, ending at ttt, and visiting no vertex twice; it is identified with its arc set P⊆EP\subseteq EP⊆E. Write Sst\mathcal S_{st}Sst​ for the finite set of these paths.

The weighted single-commodity game has a finite set of players; player iii has a weight wi≥1w_i\ge1wi​≥1, arc eee has a fixed cost ce≥0c_e\ge0ce​≥0, and every player's strategy set is Sst\mathcal S_{st}Sst​. In a profile S=(Si)iS=(S_i)_iS=(Si​)i​ let

We=∑i : e∈SiwiW_e=\sum_{i\,:\,e\in S_i} w_iWe​=i:e∈Si​∑​wi​

be the total weight on arc eee. Player iii pays

payi(S)=∑e∈SiwiWe ce.\mathrm{pay}_i(S)=\sum_{e\in S_i}\frac{w_i}{W_e}\,c_e .payi​(S)=e∈Si​∑​We​wi​​ce​.

A profile is a (pure) Nash equilibrium if no player can lower its payment by switching alone to another path.

A best-response move of player iii replaces SiS_iSi​ by a path TTT that minimises iii's payment given the other players' paths, provided this strictly lowers iii's payment. Best-response dynamics is any sequence of profiles in which each profile arises from the previous one by a best-response move of some player.

Formalization targets

Goal: Theorem 6.3 (p. 1620)

For every such game with wi≥1w_i\ge1wi​≥1 and ce≥0c_e\ge0ce​≥0:

there is no infinite sequence S0,S1,… with Sn+1 a best-response move from Sn;\text{there is no infinite sequence } S^0,S^1,\dots \text{ with } S^{n+1} \text{ a best-response move from } S^n;there is no infinite sequence S0,S1,… with Sn+1 a best-response move from Sn; a profile admitting no best-response move is a Nash equilibrium;\text{a profile admitting no best-response move is a Nash equilibrium;}a profile admitting no best-response move is a Nash equilibrium; Sst≠∅  ⟹  a pure Nash equilibrium exists.\mathcal S_{st}\neq\emptyset \;\Longrightarrow\; \text{a pure Nash equilibrium exists.}Sst​=∅⟹a pure Nash equilibrium exists.

The goal asserts only termination and existence; it fixes no bound on the length of a run.

Milestones (proof of Theorem 6.3, p. 1620)

For a profile SSS define the marginal cost of a path, cS(P)=∑e∈Pce/We∈[0,+∞]c_S(P)=\sum_{e\in P}c_e/W_e\in[0,+\infty]cS​(P)=∑e∈P​ce​/We​∈[0,+∞], and the tuple P(S)P(S)P(S) of all values cS(P)c_S(P)cS​(P), P∈SstP\in\mathcal S_{st}P∈Sst​, sorted increasingly. With strictly positive arc costs:

  1. a player on path PPP pays wi cS(P)w_i\,c_S(P)wi​cS​(P) (this one needs only ce≥0c_e\ge0ce​≥0);
  2. inequality (6.1): if player iii makes a best-response move from P1P_1P1​ to P2P_2P2​ and P\mathcal PP is the set of paths sharing an arc with P1∪P2P_1\cup P_2P1​∪P2​, then min⁡P∈PcS′(P)<min⁡P∈PcS(P)\min_{P\in\mathcal P}c_{S'}(P)<\min_{P\in\mathcal P}c_S(P)minP∈P​cS′​(P)<minP∈P​cS​(P);
  3. every best-response move strictly decreases P(S)P(S)P(S) in the lexicographic order.

Significance

The theorem gives a guarantee about dynamics, not only about existence: in single-commodity weighted network design, any order in which players take turns playing best responses reaches a stable outcome in finitely many steps. This places the single-commodity case on the positive side of the boundary drawn by the nonexistence examples for general weighted games. The tuple of sorted path costs is a potential that is not a single number, a device that applies to other games without an exact potential.

The result is proved in the paper; it has no machine-checked proof that this mission is aware of. A formal development contributes a reusable layer for weighted cost-sharing games (payments, best responses, Nash equilibria on arbitrary strategy families), a treatment of simple directed paths in multigraphs as strategy sets, and a lexicographic termination argument over sorted lists of extended reals. The goal is stated for nonnegative costs, as in the paper's model, while the printed proof uses positive costs; closing that gap is part of the work.

Difficulty

The obvious route, finding a real-valued function that every improving move decreases, is unavailable: the paper notes that Rosenthal's potential Φ\PhiΦ is not a potential once weights are added, and that improving moves can increase it. Termination must instead come from an ordinal quantity, a whole sorted list compared lexicographically, and the move of one player changes the marginal costs of every path that shares an arc with the old or the new route, in both directions.

The argument also depends on the shape of the strategy sets. Two distinct simple sss–ttt paths are never nested as arc sets; with walks that repeat vertices, or with arbitrary strategy families, the comparison between a path's marginal cost before and after a deviation can fail. Arcs of cost zero create a further gap: ce/Wec_e/W_ece​/We​ is 0/00/00/0 on an unused free arc, and the strict inequalities of the proof degenerate, so the nonnegative-cost goal needs more than the printed argument.

Formalization scope

  • Players form a finite type; arcs form a finite type with tail and head maps into a vertex type. Parallel arcs are kept.
  • A strategy is a Finset of arcs; the strategy family of every player is the finite set of arc sets of simple sss–ttt paths (a list of consecutive arcs with distinct visited vertices). There are no paths when s=ts=ts=t.
  • Weights and costs are real numbers with wi≥1w_i\ge1wi​≥1, ce≥0c_e\ge0ce​≥0 (the predicate IsStandard); the milestones (6.1) and the lexicographic decrease assume ce>0c_e>0ce​>0.
  • Payments are real; on every used arc We≥wi≥1W_e\ge w_i\ge1We​≥wi​≥1, so the division is never by zero.
  • The marginal cost cS(P)c_S(P)cS​(P) is valued in [0,+∞][0,+\infty][0,+∞] (ℝ≥0∞): an unused arc of positive cost contributes +∞+\infty+∞. Computing it in the reals, where x/0=0x/0=0x/0=0, would make unused paths free and the milestones false.
  • Termination is the well-foundedness of the relation "S′S'S′ is reached from SSS by one best-response move" with S′S'S′ below SSS; the reverse orientation is a different statement.
  • A best-response move requires a strict improvement and an exact minimiser; dropping either makes termination trivially true or false, and the second clause of the goal (no move possible implies Nash) guards against a move relation that is too narrow.

Contributions welcome: lemmas on simple paths in multigraphs (non-nestedness), the multiset-to-sorted-list lexicographic comparison, and the treatment of zero-cost arcs.

Selected references

  • E. Anshelevich, A. Dasgupta, J. Kleinberg, É. Tardos, T. Wexler, T. Roughgarden, The Price of Stability for Network Design with Fair Cost Allocation, SIAM Journal on Computing 38(4):1602–1623, 2008. https://doi.org/10.1137/070680096
  • R. W. Rosenthal, A class of games possessing pure-strategy Nash equilibria, International Journal of Game Theory 2:65–67, 1973. https://doi.org/10.1007/BF01737559
  • D. Monderer, L. S. Shapley, Potential games, Games and Economic Behavior 14:124–143, 1996. https://doi.org/10.1006/game.1996.0044
  • H. Chen, T. Roughgarden, Network design with weighted players, Proceedings of the 18th ACM Symposium on Parallelism in Algorithms and Architectures (SPAA), 2006, pp. 28–37.
6 thms2 active usersReviewed
PreviousNext

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paper
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me