Prove2Me
Navigate
DiscoverCollectionsFormalpediaBlogsUsersMomentumMy Missions+
Prove2Me
⌕
Log in

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

Operations Research

1,339 missions · 637 completed

The discipline of applying mathematical analysis to complex decision problems in operations: allocating scarce resources, scheduling, routing, inventory, and the design of service and production systems. Drawing on mathematical programming, stochastic modeling, queueing, simulation, and game-theoretic reasoning, it seeks policies that perform provably well in systems shaped by constraints, congestion, and uncertainty.

Missions

Open702Completed637All1339
ProbabilityStatistics·Captain: mikedeng1

Simulation Output Analysis Using Standardized Time Series 3: No g in 𝓜 Makes g(σB) Almost Surely Constant, So the Length Bound Is Not AttainedResearch Paper

Motivation

A steady-state simulation produces a single long output path Y={Y(t):t≥0}Y=\{Y(t):t\ge0\}Y={Y(t):t≥0}, and the quantity of interest is its long-run mean μ\muμ. The central difficulty of simulation output analysis is that the observations are correlated, so the classical confidence interval Yˉn(1)±z σ^/n\bar Y_n(1)\pm z\,\hat\sigma/\sqrt nYˉn​(1)±zσ^/n​ needs a consistent estimate of the time-average variance constant σ2\sigma^2σ2, which is hard to obtain. The method of standardized time series (STS), introduced by Schruben (1983) and put on a general footing by Glynn and Iglehart (Math. Oper. Res. 15 (1990)), avoids estimating σ\sigmaσ: it divides the centred sample mean by a functional ggg of the whole sample path that scales like σ\sigmaσ, so that σ\sigmaσ cancels in the limit. Batch means, the area estimator and other procedures used in simulation software are all instances of this construction.

Cancelling σ\sigmaσ has a price. Glynn and Iglehart show (their Corollary 4.16) that every STS interval has asymptotic expected length at least 2σΦ−1(1−δ/2)/n2\sigma\Phi^{-1}(1-\delta/2)/\sqrt n2σΦ−1(1−δ/2)/n​, the length of the interval that knows σ\sigmaσ, and (4.23) that this bound is the infimum over all admissible ggg. This mission formalizes the next question the paper asks and answers: is the infimum attained? Proposition 4.26 says it is not, and the companion results quantify the residual randomness of the interval length.

Setting

C[0,1]C[0,1]C[0,1] is the space of continuous real functions on [0,1][0,1][0,1] with the uniform metric ρ(x,y)=sup⁡t∣x(t)−y(t)∣\rho(x,y)=\sup_t|x(t)-y(t)|ρ(x,y)=supt​∣x(t)−y(t)∣ and its Borel σ\sigmaσ-algebra; k(t)=tk(t)=tk(t)=t, and C0[0,1]={x∈C[0,1]:x(0)=0}C_0[0,1]=\{x\in C[0,1]:x(0)=0\}C0​[0,1]={x∈C[0,1]:x(0)=0}. For g:C[0,1]→Rg:C[0,1]\to\mathbb Rg:C[0,1]→R, D(g)D(g)D(g) is the set of points at which ggg is not continuous. On a probability space (Ω,F,P)(\Omega,\mathcal F,P)(Ω,F,P), BBB is a standard Brownian motion, viewed as a random element of C[0,1]C[0,1]C[0,1].

Assumption (2.1). There are finite constants μ\muμ and σ>0\sigma>0σ>0 such that Xn⇒σBX_n\Rightarrow\sigma BXn​⇒σB in C[0,1]C[0,1]C[0,1], where

Xn(t)=n1/2(Yˉn(t)−μt),Yˉn(t)=1n∫0ntY(s) ds,0≤t≤1.X_n(t)=n^{1/2}\big(\bar Y_n(t)-\mu t\big),\qquad \bar Y_n(t)=\frac1n\int_0^{nt}Y(s)\,ds,\quad 0\le t\le1 .Xn​(t)=n1/2(Yˉn​(t)−μt),Yˉn​(t)=n1​∫0nt​Y(s)ds,0≤t≤1.

The class M\mathcal MM (2.3) consists of the measurable g:C[0,1]→Rg:C[0,1]\to\mathbb Rg:C[0,1]→R with

  1. g(ax)=a g(x)g(ax)=a\,g(x)g(ax)=ag(x) for a>0a>0a>0;
  2. g(x−βk)=g(x)g(x-\beta k)=g(x)g(x−βk)=g(x) for β∈R\beta\in\mathbb Rβ∈R;
  3. P{g(B)>0}=1P\{g(B)>0\}=1P{g(B)>0}=1;
  4. P{B∈D(g)}=0P\{B\in D(g)\}=0P{B∈D(g)}=0.

With H(x)=P{B(1)/g(B)≤x}H(x)=P\{B(1)/g(B)\le x\}H(x)=P{B(1)/g(B)≤x} and α,β\alpha,\betaα,β chosen so that H(β)−H(α)=1−δH(\beta)-H(\alpha)=1-\deltaH(β)−H(α)=1−δ, the interval [Yˉn(1)−g(Yˉn)β, Yˉn(1)−g(Yˉn)α][\bar Y_n(1)-g(\bar Y_n)\beta,\ \bar Y_n(1)-g(\bar Y_n)\alpha][Yˉn​(1)−g(Yˉn​)β, Yˉn​(1)−g(Yˉn​)α] (2.10) is an asymptotic 100(1−δ)%100(1-\delta)\%100(1−δ)% confidence interval for μ\muμ, of width Ln=g(Yˉn)(β−α)L_n=g(\bar Y_n)(\beta-\alpha)Ln​=g(Yˉn​)(β−α).

Formalization targets

Goal: Proposition 4.26

For every σ>0\sigma>0σ>0 there is no g∈Mg\in\mathcal Mg∈M such that, for some α>0\alpha>0α>0,

P{g(σB)=ασ}=1.(4.25)P\{g(\sigma B)=\alpha\sigma\}=1. \qquad (4.25)P{g(σB)=ασ}=1.(4.25)

By the paper's discussion before (4.25), attaining the lower bound of Corollary 4.16 with some g∈Mg\in\mathcal Mg∈M requires (4.25); the goal therefore says that the bound is attained by no g∈Mg\in\mathcal Mg∈M. The goal is stated in terms of (4.25) and not of the limit (4.24), so it does not depend on the expected-length machinery of Corollary 4.16.

Milestones (the steps of the proof, pp. 13–14)

  1. For ∣z∣<η|z|<\eta∣z∣<η: P{∣B(t)−z∣<η, max⁡0≤s≤t∣B(s)∣<2η}>0P\{|B(t)-z|<\eta,\ \max_{0\le s\le t}|B(s)|<2\eta\}>0P{∣B(t)−z∣<η, max0≤s≤t​∣B(s)∣<2η}>0.
  2. (4.27): P{ρ(σB,x)<ε}>0P\{\rho(\sigma B,x)<\varepsilon\}>0P{ρ(σB,x)<ε}>0 for every x∈C0[0,1]x\in C_0[0,1]x∈C0​[0,1] and ε>0\varepsilon>0ε>0.
  3. If the range of ggg over every ε\varepsilonε-neighbourhood of xxx contains {ασ:σ>0}\{\alpha\sigma:\sigma>0\}{ασ:σ>0}, then x∈D(g)x\in D(g)x∈D(g).
  4. Under (2.3i) and (4.25), C0[0,1]⊆D(g)C_0[0,1]\subseteq D(g)C0​[0,1]⊆D(g).

Companions

  • g(B)g(B)g(B) is nondegenerate for every g∈Mg\in\mathcal Mg∈M (p. 14).
  • Proposition 4.32: if {g2(Xn)}\{g^2(X_n)\}{g2(Xn​)} is uniformly integrable, then
lim⁡n→∞nE(Ln−ELn)2=σ2E(g(B)−Eg(B))2(β−α)2>0.\lim_{n\to\infty}nE(L_n-EL_n)^2=\sigma^2E\big(g(B)-Eg(B)\big)^2(\beta-\alpha)^2>0 .n→∞lim​nE(Ln​−ELn​)2=σ2E(g(B)−Eg(B))2(β−α)2>0.

Significance

The result closes the expected-length analysis of STS intervals: the bound 2σΦ−1(1−δ/2)2\sigma\Phi^{-1}(1-\delta/2)2σΦ−1(1−δ/2) is the exact infimum over M\mathcal MM but no single standardization reaches it, so every STS procedure is strictly worse, in expected length, than the interval with known σ\sigmaσ. The companion results explain why: g(B)g(B)g(B) is never degenerate, so n1/2Lnn^{1/2}L_nn1/2Ln​ converges to a nondegenerate random limit and the interval length fluctuates at order n−1/2n^{-1/2}n−1/2, while intervals built on a consistent estimator of σ\sigmaσ (the regenerative method, Proposition 4.34) fluctuate only at order n−1n^{-1}n−1. This is the quantitative trade-off a practitioner faces when choosing between STS and variance-estimation methods.

The paper's results are proved; none of them is formalized. The mission produces machine-checked versions of a support property of Wiener measure on C[0,1]C[0,1]C[0,1] (every path started at 000 is in the support), which is reusable well beyond simulation, and of the deterministic step that turns such a support property into everywhere-discontinuity of a functional.

Difficulty

The goal's statement is elementary, but its proof needs a quantitative fact about Brownian paths: the law of σB\sigma BσB charges every uniform ball around every path in C0[0,1]C_0[0,1]C0​[0,1]. The natural first attempt, using that ggg is continuous PPP-almost everywhere and constant almost surely on σB\sigma BσB, fails without this fact, because almost-sure statements say nothing about any particular path xxx. Mathlib provides Brownian finite-dimensional laws and independent increments, but no small-ball or support estimate for Brownian motion in the uniform metric, so (4.27) has to be built from scratch. A second subtlety is that (4.25) is given for one σ\sigmaσ only; the homogeneity (2.3i) is what upgrades it to all scales, and that upgrade is what makes the range of ggg near xxx unbounded.

Proposition 4.32 needs, in addition, the passage from weak convergence of g(Xn)g(X_n)g(Xn​) to convergence of second moments under uniform integrability, in the space C[0,1]C[0,1]C[0,1].

Formalization scope

  • C[0,1]C[0,1]C[0,1] is C(unitInterval, ℝ) with its sup norm; ρ\rhoρ is dist. Mathlib has no measurable structure on it at this commit, so the Borel σ\sigmaσ-algebra is declared in the mission's definitions file.
  • BBB is a measurable map Ω→C[0,1]\Omega\to C[0,1]Ω→C[0,1] whose coordinates agree, for every ω\omegaω, with a process satisfying Mathlib's IsBrownianReal. "Measurable" in M\mathcal MM means Borel measurable.
  • D(g)D(g)D(g) is the set where ggg is not continuous; P{B∈D(g)}P\{B\in D(g)\}P{B∈D(g)} is the outer measure of the preimage, so no measurability of D(g)D(g)D(g) is assumed.
  • Assumption (2.1) is a structure: joint measurability of YYY, local integrability of its paths (the implicit condition that makes Yˉn\bar Y_nYˉn​ defined), Yˉn\bar Y_nYˉn​ pinned pointwise by its integral formula, σ>0\sigma>0σ>0, and convergence in distribution of XnX_nXn​ to σB\sigma BσB. The index nnn ranges over N\mathbb NN.
  • P{⋅}=1P\{\cdot\}=1P{⋅}=1 is "almost surely". 0<δ<10<\delta<10<δ<1 is added where the confidence level appears.
  • The paper writes "D(g)=C0[0,1]D(g)=C_0[0,1]D(g)=C0​[0,1]" at the end of the proof of Proposition 4.26; its argument proves only C0[0,1]⊆D(g)C_0[0,1]\subseteq D(g)C0​[0,1]⊆D(g), which is what milestone 4 states.
  • Proposition 4.32 uses Bochner integrals; its uniform-integrability hypothesis makes every integral there finite.

The goal is a negation of an existence statement, so it would be trivially true if M\mathcal MM were empty or if the Brownian hypothesis were unsatisfiable. Neither holds: M\mathcal MM contains the batch-means functionals of the paper's Example 3.1 (p. 5), and the goal assumes nothing about small balls, (4.27) or continuity of ggg. Those appear only as milestones.

Contributions welcome: the support theorem for Brownian motion in C[0,1]C[0,1]C[0,1] (milestones 1–2), any of the deterministic steps, and the moment-convergence argument of Proposition 4.32.

Selected references

  • P. W. Glynn and D. L. Iglehart, Simulation output analysis using standardized time series, Mathematics of Operations Research 15(1):1–16, 1990. https://doi.org/10.1287/moor.15.1.1
  • L. Schruben, Confidence interval estimation using standardized time series, Operations Research 31(6):1090–1108, 1983. https://doi.org/10.1287/opre.31.6.1090
  • P. Billingsley, Convergence of Probability Measures, Wiley, 1968. https://doi.org/10.1002/9780470316962
8 thms1 active userReviewed
Bandit AlgorithmsMachine LearningProbability·Captain: mikedeng1

MNL-Bandit: A Dynamic Learning Approach to Assortment Selection III: On Instances Separated by Δ(v) the UCB Policy Has Regret at Most B₁N² log T/Δ(v) + B₂Research Paper

Motivation

A retailer with limited shelf or screen space must choose which subset of its products to show each arriving customer. Which assortment earns the most depends on customer preferences, and those are unknown at first: the seller learns them from the purchases it observes. This is dynamic assortment selection, and it is a standard model in online retail and revenue management. Agrawal, Avadhanula, Goyal and Zeevi (Oper. Res. 67(5), 2019; preprint arXiv:1706.03880v2) study it under the multinomial logit (MNL) choice model as the MNL-Bandit problem. They propose an epoch-based upper-confidence-bound policy (Algorithm 1) that needs no prior knowledge of the parameters.

Their worst-case guarantee (Theorem 1) is regret of order NTlog⁡NT\sqrt{NT\log NT}NTlogNT​. Many instances are easier than the worst case: the best assortment beats every other feasible assortment by a fixed margin. For such well-separated instances, earlier policies obtained logarithmic regret, but only when told a lower bound on that margin:

  • Rusmevichientong, Shen and Shmoys (2010), O(N2log⁡2T)O(N^2\log^2 T)O(N2log2T);
  • Sauré and Zeevi (2013), O(Nlog⁡T)O(N\log T)O(NlogT) for fixed cardinality.

This mission formalizes the paper's §6.1, which shows that the same parameter-free Algorithm 1 adapts to the separation.

Setting

There are NNN products with known revenues ri∈[0,1]r_i\in[0,1]ri​∈[0,1] and unknown attraction parameters vi≥0v_i\ge0vi​≥0. The no-purchase option has weight v0=1v_0=1v0​=1. Offered an assortment SSS, a customer buys i∈Si\in Si∈S with probability vi/(1+∑j∈Svj)v_i/(1+\sum_{j\in S}v_j)vi​/(1+∑j∈S​vj​) and buys nothing with probability 1/(1+∑j∈Svj)1/(1+\sum_{j\in S}v_j)1/(1+∑j∈S​vj​) (2.1). The expected revenue of SSS is

R(S,v)=∑i∈Srivi1+∑j∈Svj(2.2).R(S,\mathbf v)=\frac{\sum_{i\in S}r_iv_i}{1+\sum_{j\in S}v_j}\quad(2.2).R(S,v)=1+∑j∈S​vj​∑i∈S​ri​vi​​(2.2).

Feasible assortments form a family S\mathcal SS defined by totally unimodular constraints A x(S)≤bA\,x(S)\le bAx(S)≤b (2.3). Assumption 4.1 asks that vi≤v0=1v_i\le v_0=1vi​≤v0​=1 for all iii and that S\mathcal SS be closed under taking subsets.

A policy chooses St∈SS_t\in\mathcal SSt​∈S from the past choices. The regret after TTT customers is Regπ(T,v)=T R(S∗,v)−Eπ∑t=1TR(St,v)\mathrm{Reg}_\pi(T,\mathbf v)=T\,R(S^*,\mathbf v)-\mathbb E_\pi\sum_{t=1}^TR(S_t,\mathbf v)Regπ​(T,v)=TR(S∗,v)−Eπ​∑t=1T​R(St​,v), where S∗S^*S∗ maximizes R(⋅,v)R(\cdot,\mathbf v)R(⋅,v) over S\mathcal SS (2.6).

Algorithm 1 works in epochs: epoch ℓ\ellℓ offers one assortment SℓS_\ellSℓ​ until a customer buys nothing. It keeps three statistics for each product iii:

  • Ti(ℓ)T_i(\ell)Ti​(ℓ), the number of epochs up to ℓ\ellℓ that offered iii;
  • vˉi,ℓ\bar v_{i,\ell}vˉi,ℓ​, the average number of purchases of iii per such epoch;
  • the upper confidence bound vi,ℓUCB=vˉi,ℓ+vˉi,ℓ 48log⁡(Nℓ+1)/Ti(ℓ)+48log⁡(Nℓ+1)/Ti(ℓ)v^{\mathrm{UCB}}_{i,\ell}=\bar v_{i,\ell}+\sqrt{\bar v_{i,\ell}\,48\log(\sqrt N\ell+1)/T_i(\ell)}+48\log(\sqrt N\ell+1)/T_i(\ell)vi,ℓUCB​=vˉi,ℓ​+vˉi,ℓ​48log(N​ℓ+1)/Ti​(ℓ)​+48log(N​ℓ+1)/Ti​(ℓ).

The next epoch offers an assortment maximizing the MNL revenue computed with vUCBv^{\mathrm{UCB}}vUCB in place of v\mathbf vv.

The separation of an instance is the gap between the optimal and the second-best expected revenue among feasible assortments,

Δ(v)=min⁡S∈S: R(S,v)≠R(S∗,v)(R(S∗,v)−R(S,v))(6.1).\Delta(\mathbf v)=\min_{S\in\mathcal S:\,R(S,\mathbf v)\neq R(S^*,\mathbf v)}\big(R(S^*,\mathbf v)-R(S,\mathbf v)\big)\quad(6.1).Δ(v)=S∈S:R(S,v)=R(S∗,v)min​(R(S∗,v)−R(S,v))(6.1).

The analysis uses the threshold τ=4NClog⁡NT/Δ2(v)\tau=4NC\log NT/\Delta^2(\mathbf v)τ=4NClogNT/Δ2(v) (6.2), with C=max⁡{C12,C2}C=\max\{C_1^2,C_2\}C=max{C12​,C2​}, C1=72+24C_1=\sqrt{72}+\sqrt{24}C1​=72​+24​ and C2=144C_2=144C2​=144. It also uses good epochs: those in which every vi,ℓUCBv^{\mathrm{UCB}}_{i,\ell}vi,ℓUCB​ lies above viv_ivi​ and within the confidence width of Lemma 4.1.

Formalization targets

Goal: Theorem 3 (p. 20)

There are absolute constants B1,B2B_1,B_2B1​,B2​ such that for every instance satisfying Assumption 4.1 with ri∈[0,1]r_i\in[0,1]ri​∈[0,1], every tie-breaking rule and every TTT,

Regπ(T,v)≤B1 N2log⁡TΔ(v)+B2.\mathrm{Reg}_\pi(T,\mathbf v)\le B_1\,\frac{N^2\log T}{\Delta(\mathbf v)}+B_2.Regπ​(T,v)≤B1​Δ(v)N2logT​+B2​.

The constants are not fixed, so the goal survives any improvement of them.

Milestones

  • Lemma 4.1 (p. 13): vi,ℓUCB≥viv^{\mathrm{UCB}}_{i,\ell}\ge v_ivi,ℓUCB​≥vi​ with probability ≥1−6/(Nℓ)\ge1-6/(N\ell)≥1−6/(Nℓ), and vi,ℓUCB−vi≤C1vilog⁡(Nℓ+1)/Ti(ℓ)+C2log⁡(Nℓ+1)/Ti(ℓ)v^{\mathrm{UCB}}_{i,\ell}-v_i\le C_1\sqrt{v_i\log(\sqrt N\ell+1)/T_i(\ell)}+C_2\log(\sqrt N\ell+1)/T_i(\ell)vi,ℓUCB​−vi​≤C1​vi​log(N​ℓ+1)/Ti​(ℓ)​+C2​log(N​ℓ+1)/Ti​(ℓ) with probability ≥1−7/(Nℓ)\ge1-7/(N\ell)≥1−7/(Nℓ).
  • Lemma 4.3 (p. 14): (1+∑j∈Svj)(R~(S)−R(S,v))(1+\sum_{j\in S}v_j)(\tilde R(S)-R(S,\mathbf v))(1+∑j∈S​vj​)(R~(S)−R(S,v)) is at most the sum of these widths over the offered set, with probability ≥1−13/ℓ\ge1-13/\ell≥1−13/ℓ.
  • Lemma 6.1 (p. 20): in a good epoch, if every offered product has Ti(ℓ)≥τT_i(\ell)\ge\tauTi​(ℓ)≥τ, the offered assortment is optimal.
  • Lemma 6.2 (p. 20): at most NτN\tauNτ good epochs offer a sub-optimal assortment.
  • Corollary C.1 (p. 48): at most Nlog⁡NTN\log NTNlogNT epochs offer a product with Ti(ℓ)<log⁡NTT_i(\ell)<\log NTTi​(ℓ)<logNT.
  • (C.5) (p. 48): the regret is at most Eπ∑ℓ≤L(1+V(Sℓ))(R(S∗,v)−R(Sℓ,v))\mathbb E_\pi\sum_{\ell\le L}(1+V(S_\ell))(R(S^*,\mathbf v)-R(S_\ell,\mathbf v))Eπ​∑ℓ≤L​(1+V(Sℓ​))(R(S∗,v)−R(Sℓ​,v)).

Companion: Corollary 6.1 (p. 21)

Under the cardinality constraint ∣S∣≤K|S|\le K∣S∣≤K: Regπ(T,v)≤B1 NKlog⁡NT/Δ(v)+B2\mathrm{Reg}_\pi(T,\mathbf v)\le B_1\,NK\log NT/\Delta(\mathbf v)+B_2Regπ​(T,v)≤B1​NKlogNT/Δ(v)+B2​.

Significance

The result. Theorem 3 shows that one policy is simultaneously near-optimal in the worst case (Theorem 1, O~(NT)\tilde O(\sqrt{NT})O~(NT​), matched by the lower bound of Theorem 2 up to logarithms) and logarithmic on separated instances. It does so without being told Δ(v)\Delta(\mathbf v)Δ(v), which the separation-based policies of Rusmevichientong et al. and Sauré–Zeevi require. Under a cardinality constraint, Corollary 6.1 matches the order of Sauré–Zeevi's bound.

Formalizing it. The theorem is proved in the paper (Appendix C), and no machine-checked version exists. A formal proof has to make several things precise:

  • Algorithm 1 as a function of the observed history;
  • the epoch structure of a finite horizon;
  • the pathwise counting arguments of Lemmas 6.1–6.2 and Corollary C.1;
  • the probabilistic estimates inherited from the worst-case analysis.

Two printed details need care, and the formal statements fix them: Appendix C defines good epochs with "or" where both inequalities are meant, and Lemma 6.2's proof skips one case.

Difficulty

The counting part looks routine but depends on indexing. The bounds that are good at the end of epoch ℓ\ellℓ select the assortment of epoch ℓ+1\ell+1ℓ+1, so "a good epoch offering a sub-optimal set" pairs two consecutive epochs. A per-product count of under-sampled epochs is then off by one per product, so the printed bound NτN\tauNτ does not follow from the statement of Lemma 6.1 by counting alone.

The probabilistic part is the harder one. The per-epoch purchase counts are geometric with mean viv_ivi​ only conditionally on adaptively chosen assortments. The confidence statements must hold uniformly over a random number of samples, and the regret must be converted from customers to epochs, whose lengths are random and truncated at TTT. A union bound over all epochs costs a factor log⁡T\log TlogT, and that factor has to stay within the claimed N2log⁡T/ΔN^2\log T/\DeltaN2logT/Δ.

Formalization scope

Products are Fin N (0-based), and choices are Option (Fin N) with none the no-purchase option. A policy maps the past choices to an assortment. The law of a length-TTT history is the product of MNL probabilities, so every expectation is a finite sum. Revenue is the published ChoiceCDLP.MNL.mnlObjective v r 1, and R(S∗,v)R(S^*,\mathbf v)R(S∗,v) is a Finset.sup' over the nonempty family S\mathcal SS. Algorithm 1 is defined as an explicit state machine and never reads v\mathbf vv. Ties in its argmax are broken by an arbitrary selector, and every statement quantifies over all selectors.

Standing assumptions carried by every statement:

  • 0≤vi≤10\le v_i\le10≤vi​≤1;
  • ri∈[0,1]r_i\in[0,1]ri​∈[0,1];
  • the totally unimodular form of S\mathcal SS;
  • closure under subsets;
  • nonemptiness of S\mathcal SS (needed for S∗S^*S∗ to exist).

Disclosed conventions:

  • UCB before sampling. vi,ℓUCB=1v^{\mathrm{UCB}}_{i,\ell}=1vi,ℓUCB​=1 while Ti(ℓ)=0T_i(\ell)=0Ti​(ℓ)=0.
  • Degenerate separation. Δ(v)=1\Delta(\mathbf v)=1Δ(v)=1 when every feasible assortment is optimal; the regret is then 000.
  • Lemma 4.1 constants. C1,C2C_1,C_2C1​,C2​ take the values established in the proof of Lemma 4.1.
  • Probabilistic lemmas. They are stated for every horizon TTT as probabilities that epoch ℓ\ellℓ is completed and the bound fails.
  • Lemma 4.3. The printed free index iii is read as the sum over the offered set, and R~\tilde RR~ uses the bounds of the preceding epoch.
  • (C.5). It is stated as "≤\le≤", since the last epoch is truncated at TTT.

The constants B1,B2B_1,B_2B1​,B2​ are quantified before every other object. A constant depending on NNN, v\mathbf vv or Δ\DeltaΔ would make the theorem say only that regret is finite. The policy is pinned to Algorithm 1, not to any optimistic rule.

Contributions are welcome at every level:

  • the pathwise lemmas (6.1, 6.2, C.1), which need no probability;
  • the epoch rewriting (C.5);
  • the concentration Lemmas 4.1 and 4.3, shared with the companion mission on Theorem 1;
  • reusable infrastructure: MNL choice probabilities and finite-horizon history laws for adaptive policies.

Selected references

  • S. Agrawal, V. Avadhanula, V. Goyal, A. Zeevi, MNL-Bandit: A Dynamic Learning Approach to Assortment Selection, Operations Research 67(5):1453–1485, 2019. https://doi.org/10.1287/opre.2018.1832 ; preprint arXiv:1706.03880v2. https://arxiv.org/abs/1706.03880v2
  • P. Rusmevichientong, Z.-J. M. Shen, D. B. Shmoys, Dynamic assortment optimization with a multinomial logit choice model and capacity constraint, Operations Research 58(6):1666–1680, 2010. https://doi.org/10.1287/opre.1100.0866
  • D. Sauré, A. Zeevi, Optimal dynamic assortment planning with demand learning, Manufacturing & Service Operations Management 15(3):387–404, 2013. https://doi.org/10.1287/msom.2013.0429
  • K. Talluri, G. van Ryzin, Revenue management under a general discrete choice model of consumer behavior, Management Science 50(1):15–33, 2004. https://doi.org/10.1287/mnsc.1030.0147
9 thms1 active userReviewed
Dynamic ProgrammingProbabilityStochastic Systems·Captain: mikedeng1

Old and New Methods for Lost-Sales Inventory Systems 1: Under Any Policy in Z(s) the State Enters X(s) and Never Leaves, So Every State Outside X(s) Is TransientResearch Paper

Why the lost-sales state space matters

The lost-sales inventory system with a positive lead time is one of the basic models of inventory theory. Demand that cannot be met from stock is lost rather than backlogged, and orders arrive a fixed number LLL of periods after they are placed. Unlike the backlog system, where the inventory position is a sufficient one-dimensional state and a base-stock policy is optimal, the lost-sales system needs the whole pipeline of outstanding orders as its state, and its optimal policy has no simple form. Computing optimal policies by dynamic programming therefore runs into the size of an LLL-dimensional state space.

Paul Zipkin's paper Old and New Methods for Lost-Sales Inventory Systems (Operations Research 56(5), 2008, doi:10.1287/opre.1070.0471) revisits this model, compares classical and new heuristics against optimal policies, and prepares the computation with a state-reduction result. Morton (1969, SIAM Review 11, Equation (83)) showed that under an optimal policy the state remains in a bounded set; Zipkin's §4 extends the argument to every policy in a class Z(s)Z(s)Z(s) that contains the usual heuristics. This mission formalizes that section: the reduced state space X(s)X(s)X(s), the policy class Z(s)Z(s)Z(s), and Claim 1, which says that states outside X(s)X(s)X(s) are transient.

Setting

Fix the lead time L≥1L\ge 1L≥1. The state is the LLL-vector x=(x0,…,xL−1)x=(x_0,\dots,x_{L-1})x=(x0​,…,xL−1​): x0x_0x0​ is the stock on hand after this period's delivery, and xlx_lxl​ (1≤l≤L−11\le l\le L-11≤l≤L−1) is the order that arrives lll periods later, so xL−1x_{L-1}xL−1​ is the most recent order. Given the state xxx, an order z≥0z\ge 0z≥0 and a demand d≥0d\ge 0d≥0, the next state is

x+=([x0−d]++x1, x2, …, xL−1, z),[a]+=max⁡(a,0).x_+ = \bigl([x_0-d]^+ + x_1,\ x_2,\ \dots,\ x_{L-1},\ z\bigr),\qquad [a]^+=\max(a,0).x+​=([x0​−d]++x1​, x2​, …, xL−1​, z),[a]+=max(a,0).

Demands in successive periods are independent with a common law DDD on [0,∞)[0,\infty)[0,∞).

The pipeline partial sums are vl=∑m=lL−1xmv_l=\sum_{m=l}^{L-1}x_mvl​=∑m=lL−1​xm​ for l=0,…,L−1l=0,\dots,L-1l=0,…,L−1 and vL=0v_L=0vL​=0; v0v_0v0​ is the inventory position and vL−1=xL−1v_{L-1}=x_{L-1}vL−1​=xL−1​ the most recent order. A level vector is s=(s0,…,sL)s=(s_0,\dots,s_L)s=(s0​,…,sL​) with s0≥s1≥⋯≥sL≥0s_0\ge s_1\ge\dots\ge s_L\ge 0s0​≥s1​≥⋯≥sL​≥0. For a level vector sss,

X(s)={x≥0: vl≤sl, l=0,…,L−1},X(s)=\{x\ge 0:\ v_l\le s_l,\ l=0,\dots,L-1\},X(s)={x≥0: vl​≤sl​, l=0,…,L−1},

and Z(s)Z(s)Z(s) is the set of stationary policies zzz (an order z(x)≥0z(x)\ge 0z(x)≥0 in each state x≥0x\ge 0x≥0) with z(x)=0z(x)=0z(x)=0 for x∉X(s)x\notin X(s)x∈/X(s) and vl+z(x)≤slv_l+z(x)\le s_lvl​+z(x)≤sl​ for l=0,…,Ll=0,\dots,Ll=0,…,L and x∈X(s)x\in X(s)x∈X(s). The vector base-stock policy with parameter sss is

zs(x)=[min⁡{sl−vl, l=0,…,L}]+,z_s(x)=\bigl[\min\{s_l-v_l,\ l=0,\dots,L\}\bigr]^+,zs​(x)=[min{sl​−vl​, l=0,…,L}]+,

equation (5) of the paper with a general sss in place of Morton's sˉ\bar ssˉ. Under a stationary policy zzz, from an initial state xxx and along a demand path (dt)(d_t)(dt​), the state process is x0=xx_0=xx0​=x, xt+1=(xt)+x_{t+1}=(x_t)_+xt+1​=(xt​)+​ with order z(xt)z(x_t)z(xt​) and demand dtd_tdt​. In Lean these are next, v, IsLevelVector, Xs, Zs, vectorBaseStock and traj in the namespace LostSalesOldNew.StateReduction.

Formalization targets

Goal: Claim 1 (p. 1259)

For a level vector sss, a policy z∈Z(s)z\in Z(s)z∈Z(s), i.i.d. demands with law DDD on [0,∞)[0,\infty)[0,∞) and P(d>0)>0P(d>0)>0P(d>0)>0, and any initial state x≥0x\ge 0x≥0, almost surely

∃ T ∀ t≥T:xt∈X(s).\exists\,T\ \forall\,t\ge T:\quad x_t\in X(s).∃T ∀t≥T:xt​∈X(s).

The paper's word is "transient"; the statement above says that every state outside X(s)X(s)X(s), indeed the whole complement, is visited only finitely often.

Milestones

  1. (5) and §4: for every level vector sss, zs∈Z(s)z_s\in Z(s)zs​∈Z(s).
  2. Invariance (proof of Claim 1): x∈X(s)x\in X(s)x∈X(s), z∈Z(s)z\in Z(s)z∈Z(s), d≥0d\ge 0d≥0 imply x+∈X(s)x_+\in X(s)x+​∈X(s).
  3. Entrance (proof of Claim 1): along a demand path with dt≥0d_t\ge 0dt​≥0 and ∑t<ndt→∞\sum_{t<n}d_t\to\infty∑t<n​dt​→∞, the process started at any x≥0x\ge 0x≥0 visits X(s)X(s)X(s).

A further item records the paper's remark that X(s)X(s)X(s) and Z(s)Z(s)Z(s) are increasing in sss.

Significance

Claim 1 lets the dynamic program be solved on the compact set X(sˉ)X(\bar s)X(sˉ) with policies restricted to Z(sˉ)Z(\bar s)Z(sˉ), which is what makes the paper's exact computations (lead times up to four, §6) feasible. Since Z(s)Z(s)Z(s) contains the vector base-stock policies and, for sss large enough, any policy that stops ordering in large states, the claim also says that every plausible policy eventually confines the state to a bounded set.

The result has a short proof in the paper and no machine-checked version. Formalizing it produces a reusable encoding of the lost-sales pipeline with general stationary policies, and makes explicit two points the paper leaves implicit: the probabilistic reading of "transient", and the demand condition under which the process must leave the complement of X(s)X(s)X(s). Related platform material treats the backlog analogue (VeinottBaseStock.base_stock_level_absorbing, a base-stock level set is absorbing) and lost-sales dynamics under other conventions (CappedBaseStock_Model, from an empty initial state under capped base-stock policies with average cost); those formulations use another state convention and cost criterion and are not imported here.

Difficulty

The invariance step is algebra on partial sums. The content of the claim is the entrance step: outside X(s)X(s)X(s) the policy orders nothing, so the process can only reach X(s)X(s)X(s) through demand draining the stock and the pipeline. That requires the cumulative demand to diverge, which fails pathwise (demand identically zero leaves a state with x0>s0x_0>s_0x0​>s0​ fixed forever) and holds only almost surely, from independence and P(d>0)>0P(d>0)>0P(d>0)>0. The naive reading of "transient" as invariance alone, or as the pathwise statement with divergent demand assumed, misses this step.

Formalization scope

  • States are Fin L → ℝ with L≥1L\ge 1L≥1 in the goal; theorems quantify over nonnegative states. The vector (x,z)(x,z)(x,z) is Fin.snoc x z; vvv takes a natural-number index and vanishes at l≥Ll\ge Ll≥L, so vL=0v_L=0vL​=0.
  • Level vectors are Fin (L+1) → ℝ, antitone with nonnegative last entry. X(s)X(s)X(s) constrains v0,…,vL−1v_0,\dots,v_{L-1}v0​,…,vL−1​; Z(s)Z(s)Z(s) constrains vl+z(x)v_l+z(x)vl​+z(x) for l=0,…,Ll=0,\dots,Ll=0,…,L.
  • Policies are deterministic stationary functions of the state, with no measurability required. The zero condition and the sign condition of Z(s)Z(s)Z(s) range over states x≥0x\ge 0x≥0 (the paper's state space); over all of RL\mathbb R^LRL the vector base-stock policy would not be in Z(s)Z(s)Z(s).
  • Demand is a probability measure DDD on R\mathbb RR with D((−∞,0))=0D((-\infty,0))=0D((−∞,0))=0; the demand sequence is the coordinate process under Measure.infinitePi (fun _ : ℕ => D). The hypothesis D((0,∞))>0D((0,\infty))>0D((0,∞))>0 is added to the paper's assumptions; without it the claim is false.
  • "Transient" is read as: almost surely the state lies in X(s)X(s)X(s) from some period on.
  • Ruled out: the goal is neither the invariance statement alone, which says nothing about states outside X(s)X(s)X(s), nor the pathwise entrance statement with divergent cumulative demand assumed, which drops the probabilistic content.

A complete development needs the partial-sum algebra of the pipeline, the second Borel–Cantelli lemma or the strong law for i.i.d. nonnegative sequences under Measure.infinitePi, and induction along traj. The pipeline encoding is shared with the second mission of this series (lead-time monotonicity). Proofs of any milestone and of the goal are welcome.

Selected references

  • P. Zipkin, Old and New Methods for Lost-Sales Inventory Systems, Operations Research 56(5):1256–1263, 2008. https://doi.org/10.1287/opre.1070.0471
  • T. Morton, Bounds on the Solution of the Lagged Optimal Inventory Equation with No Demand Backlogging and Proportional Costs, SIAM Review 11:572–576, 1969 (as cited in Zipkin 2008).
  • T. Morton, The Near-Myopic Nature of the Lagged-Proportional-Cost Inventory Problem with Lost Sales, Operations Research 19(7):1708–1716, 1971. https://doi.org/10.1287/opre.19.7.1708
  • R. Levi, G. Janakiraman, M. Nagarajan, Provably Near-Optimal Balancing Policies for Stochastic Inventory Control Models with Lost Sales, Mathematics of Operations Research, 2008 (forthcoming at the time of Zipkin 2008).
5 thms1 active userReviewed
Dynamic ProgrammingOptimizationProbability·Captain: mikedeng1

On the Structure of Lost-Sales Inventory Models 1: In the Transformed State v = J⁻¹x, the Optimal Cost Functions f̄_t(v) and ḡ_t(v, ζ) Are L♮-ConvexResearch Paper

Motivation

The lost-sales inventory system is the inventory model in which demand that cannot be met from stock is lost, not backlogged. It is the natural model for retail and for any setting where a customer who finds the shelf empty buys elsewhere. With a positive order lead time it is also one of the classical hard problems of inventory theory. In the backorder model the state collapses to a single number, the inventory position, and a base-stock policy is optimal. Under lost sales the whole pipeline of outstanding orders matters, the state is an LLL-dimensional vector, and no simple policy is optimal.

Its structure has a long history:

  • Karlin and Scarf (1958) formulated the model and showed, for lead time one and continuous demand, that the optimal order decreases in on-hand inventory with slope bounded by one.
  • Morton (1969) extended these qualitative properties to general lead times, again by differentiating the optimal cost function, which required smooth demand distributions.
  • Zipkin (2008), the paper formalized here, recovered and unified both results with a change of state variables and a single structural property, L♮-convexity, from discrete convex analysis (Murota 2003). The argument needs no smoothness and works for discrete as well as continuous demand.

Setting

Time is discrete. In each period the order due arrives, a new order z≥0z\ge0z≥0 is placed, demand d≥0d\ge0d≥0 occurs, and a holding or penalty cost is charged. The lead time L≥1L\ge1L≥1 is an integer. The original state x=(x0,…,xL−1)x=(x_0,\dots,x_{L-1})x=(x0​,…,xL−1​) records on-hand inventory x0x_0x0​ followed by the orders in transit. Demands are independent with a common law μ\muμ. The costs are linear and stationary: unit procurement cost ccc, unit holding cost h^\hat hh^, unit lost-sales penalty ppp, and discount factor γ\gammaγ.

The transformed state is the vector of partial sums vl=∑k=lL−1xkv_l=\sum_{k=l}^{L-1}x_kvl​=∑k=lL−1​xk​, 0≤l<L0\le l<L0≤l<L, with the convention vL=0v_L=0vL​=0. It ranges over the cone

V={v∈RL:v0≥v1≥⋯≥vL−1≥0}.V=\{v\in\mathbb R^L : v_0\ge v_1\ge\dots\ge v_{L-1}\ge 0\}.V={v∈RL:v0​≥v1​≥⋯≥vL−1​≥0}.

The transformed order is ζ=−z≤0\zeta=-z\le0ζ=−z≤0. With eee the all-ones vector, the state moves to

v+=([v0−v1−d]++v1, v2, …, vL−1, 0)−ζe.v_+=\big([v_0-v_1-d]^+ + v_1,\ v_2,\ \dots,\ v_{L-1},\ 0\big)-\zeta e .v+​=([v0​−v1​−d]++v1​, v2​, …, vL−1​, 0)−ζe.

The one-period cost is q^(u)=h^u++pu−\hat q(u)=\hat hu^+ + pu^-q^​(u)=h^u++pu− on u=y−du=y-du=y−d, where y=v0−v1y=v_0-v_1y=v0​−v1​ is the on-hand stock, and q^0(y)=E[q^(y−d)]\hat q^0(y)=E[\hat q(y-d)]q^​0(y)=E[q^​(y−d)]. With kkk periods to go, the optimal cost functions are

fˉ0≡0,gˉk(v,ζ)=−γLcζ+q^0(v0−v1)+γE[fˉk(v+)],fˉk+1(v)=inf⁡ζ≤0gˉk(v,ζ).\bar f_0\equiv0,\qquad \bar g_k(v,\zeta)=-\gamma^Lc\zeta+\hat q^0(v_0-v_1)+\gamma E\big[\bar f_k(v_+)\big],\qquad \bar f_{k+1}(v)=\inf_{\zeta\le0}\bar g_k(v,\zeta).fˉ​0​≡0,gˉ​k​(v,ζ)=−γLcζ+q^​0(v0​−v1​)+γE[fˉ​k​(v+​)],fˉ​k+1​(v)=ζ≤0inf​gˉ​k​(v,ζ).

A function φ\varphiφ is submodular on a set S⊆RnS\subseteq\mathbb R^nS⊆Rn if φ(x∨y)+φ(x∧y)≤φ(x)+φ(y)\varphi(x\vee y)+\varphi(x\wedge y)\le\varphi(x)+\varphi(y)φ(x∨y)+φ(x∧y)≤φ(x)+φ(y) for all x,y∈Sx,y\in Sx,y∈S, with ∨,∧\vee,\wedge∨,∧ the componentwise maximum and minimum. A function f:V→Rf:V\to\mathbb Rf:V→R is L♮-convex if ψ(v,ζ)=f(v−ζe)\psi(v,\zeta)=f(v-\zeta e)ψ(v,ζ)=f(v−ζe) is submodular on V×ℜ−V\times\Re^-V×ℜ−. A function g(v,ζ)g(v,\zeta)g(v,ζ) on V×ℜ−V\times\Re^-V×ℜ− is L♮-convex if g((v,ζ)−ξ(e,1))g\big((v,\zeta)-\xi(e,1)\big)g((v,ζ)−ξ(e,1)) is submodular in (v,ζ,ξ)(v,\zeta,\xi)(v,ζ,ξ) over v∈Vv\in Vv∈V, ζ≤ξ≤0\zeta\le\xi\le0ζ≤ξ≤0.

Formalization targets

Goal: Theorem 4 (p. 939)

For every number of periods to go kkk,

fˉk is L♮-convex on Vandgˉk is L♮-convex on V×ℜ−.\bar f_k \text{ is L♮-convex on } V \quad\text{and}\quad \bar g_k \text{ is L♮-convex on } V\times\Re^- .fˉ​k​ is L♮-convex on Vandgˉ​k​ is L♮-convex on V×ℜ−.

The statement fixes no constants. It holds for every lead time, every demand law in the class below, and every cost vector.

Milestones, in the order the proof uses them

  1. Lemma 1 (p. 938): if f(v)f(v)f(v) is L♮-convex, so is ψ(v,ζ)=f(v−ζe)\psi(v,\zeta)=f(v-\zeta e)ψ(v,ζ)=f(v−ζe).
  2. Lemma 2 (p. 939): if g(v,ζ)g(v,\zeta)g(v,ζ) is L♮-convex, so is f(v)=min⁡ζ≤0g(v,ζ)f(v)=\min_{\zeta\le0}g(v,\zeta)f(v)=minζ≤0​g(v,ζ).
  3. Optimal filling (proof of Theorem 4): in the end-of-period program (3), filling as much demand as possible is optimal (a=min⁡{d,v0−v1}a=\min\{d,v_0-v_1\}a=min{d,v0​−v1​}). Consequently κˉt(v,ζ∣d)=q^(v0−v1−d)+γfˉt+1(v+)\bar\kappa_t(v,\zeta\mid d)=\hat q(v_0-v_1-d)+\gamma\bar f_{t+1}(v_+)κˉt​(v,ζ∣d)=q^​(v0​−v1​−d)+γfˉ​t+1​(v+​).
  4. ψt\psi_tψt​ is L♮-convex: the objective ψt(v+,v,ζ)=h^(v+−v1)+p(v+−v0+d)+γfˉt+1[(v+,v2,…,vL−1,0)−ζe]\psi_t(v_+,v,\zeta)=\hat h(v_+-v_1)+p(v_+-v_0+d)+\gamma\bar f_{t+1}[(v_+,v_2,\dots,v_{L-1},0)-\zeta e]ψt​(v+​,v,ζ)=h^(v+​−v1​)+p(v+​−v0​+d)+γfˉ​t+1​[(v+​,v2​,…,vL−1​,0)−ζe] of program (4) is L♮-convex on its constraint set.
  5. κˉt\bar\kappa_tκˉt​ is L♮-convex: the end-of-period cost κˉt(v,ζ∣d)\bar\kappa_t(v,\zeta\mid d)κˉt​(v,ζ∣d) is L♮-convex in (v,ζ)(v,\zeta)(v,ζ) for each ddd.
  6. gˉt\bar g_tgˉ​t​ is L♮-convex: if fˉt+1\bar f_{t+1}fˉ​t+1​ is L♮-convex, so is gˉt=−γLcζ+Ed[κˉt]\bar g_t=-\gamma^Lc\zeta+E_d[\bar\kappa_t]gˉ​t​=−γLcζ+Ed​[κˉt​], by (5).

Significance

The result. L♮-convexity of fˉt\bar f_tfˉ​t​ implies that fˉt\bar f_tfˉ​t​ is submodular and convex in the transformed state. L♮-convexity of gˉt\bar g_tgˉ​t​ yields, through Lemma 3 and Corollary 5 of the paper, that the optimal order zˉt(v)\bar z_t(v)zˉt​(v) is nonincreasing in vvv and changes by at most ω\omegaω when every component of vvv increases by ω\omegaω. These are exactly the Karlin–Scarf and Morton sensitivity results, obtained without derivatives. The same property underlies the paper's later results: the bounds on the optimal policy (§4), the monotonicity of cost in demand variability (§5), and the extensions to capacity limits, correlated demand and stochastic lead times (§6).

Formalizing it. The theorem is proved in the paper; nothing here is open mathematically. To our knowledge no machine-checked proof of L♮-convexity of an inventory value function exists. The mission produces three things:

  • a formal L♮-convexity theory on real cones (the shift characterization, its preservation under partial minimization, and its preservation under expectation);
  • a checked version of the induction behind Theorem 4;
  • a verified model of the lost-sales recursion in the transformed state, which missions on Corollary 5 and the later sections can import.

Difficulty

The obvious argument works with ordinary convexity and submodularity in the original state xxx. It stalls: the lost-sales transition [x0−d]+[x_0-d]^+[x0​−d]+ is not linear, and the order enters only the last pipeline component, so the comparative statics Karlin and Scarf and Morton obtained by differentiation do not follow from convexity in xxx. The paper's structure appears only in the partial sums vvv. Submodularity alone, moreover, is not preserved by the step that minimizes over the order. The shift structure of L♮-convexity is what survives the minimization over ζ\zetaζ (Lemma 2). Proving plain submodularity of fˉt\bar f_tfˉ​t​ on VVV therefore does not close the induction.

A second difficulty is the end-of-period program. Writing the next state as the minimum of a program over a constraint set that is a sublattice requires checking the order-theoretic structure of the set together with the sign pattern of every term. The equality between the program's value and the recursion's one-period cost (milestone 3) is a property of the model's own value function, not of an arbitrary L♮-convex continuation.

A third difficulty is measure-theoretic. Preservation under expectation needs integrability and measurability of d↦fˉk(v+)d\mapsto\bar f_k(v_+)d↦fˉ​k​(v+​), and these come from the model rather than from the definitions.

Formalization scope

  • Representation. States are functions Fin L → ℝ, indexed 0,…,L−10,\dots,L-10,…,L−1 as in the paper. A helper returns vlv_lvl​ for l<Ll<Ll<L and 000 at l=Ll=Ll=L. A function g(v,ζ)g(v,\zeta)g(v,ζ) is encoded on Fin (L+1) → ℝ with ζ\zetaζ last. The objective ψt\psi_tψt​ of (4) is encoded on Fin (L+2) → ℝ with coordinates (v+,v0,…,vL−1,ζ)(v_+,v_0,\dots,v_{L-1},\zeta)(v+​,v0​,…,vL−1​,ζ). The order is componentwise.
  • L♮-convexity on a domain DDD: (x,ξ)↦f(x−ξe)(x,\xi)\mapsto f(x-\xi e)(x,ξ)↦f(x−ξe) is submodular on {ξ≤0, x∈D, x−ξe∈D}\{\xi\le0,\ x\in D,\ x-\xi e\in D\}{ξ≤0, x∈D, x−ξe∈D}, with eee the all-ones vector on every coordinate. For D=VD=VD=V this is the paper's definition verbatim. This is the paper's definition, not Murota's translation-submodularity form, and it is stated on real vectors. The platform's discrete-convexity library (DiscreteConvex.LConvexFunctions.*) works on ZV\mathbb Z^VZV with values in R∪{+∞}\mathbb R\cup\{+\infty\}R∪{+∞}, a different domain and definition, and is not reused.
  • Time. Data are stationary, so the paper's fˉt\bar f_tfˉ​t​ depends on ttt only through the number of periods to go k=T+L+1−tk=T+L+1-tk=T+L+1−t, and "for all ttt" is "for all kkk". The minimum over z≥0z\ge0z≥0 is taken in every period, as printed in (1).
  • Explicit readings of the paper's phrases.
    • "min" (over ζ≤0\zeta\le0ζ≤0, over v+v_+v+​, over (a,w)(a,w)(a,w)) is an infimum over the constraint set. In the model every cost is nonnegative, so the infimum is genuine. The generic Lemma 2 assumes the values bounded below, and the generic milestone 5 assumes the continuation nonnegative on VVV.
    • "the dtd_tdt​ are independent and nonnegative": one law μ\muμ with μ(−∞,0)=0\mu(-\infty,0)=0μ(−∞,0)=0, since data are stationary.
    • "unit cost", "discount rate": c,h^,p≥0c,\hat h,p\ge0c,h^,p≥0 and 0<γ≤10<\gamma\le10<γ≤1.
    • Finite mean demand is added: without it q^0\hat q^0q^​0 is infinite.
    • "the optimal a=min⁡{d,v0−v1}a=\min\{d,v_0-v_1\}a=min{d,v0​−v1​}": stated as the equality of the infimum of program (4) with its value at v+=max⁡(v0−d,v1)v_+=\max(v_0-d,v_1)v+​=max(v0​−d,v1​).
  • Ruled out. A statement that holds because a value function is the junk constant 000 (constants are L♮-convex), plain submodularity on VVV in place of the shift condition, quantification over all of RL\mathbb R^LRL instead of VVV, and a goal about an arbitrary function assumed to satisfy the recursion. None of these is the paper's theorem; the goal is stated for the recursion itself.
  • Not in scope. "L♮-convex implies convex" for a generic function, which is false without regularity (a non-measurable additive function is L♮-convex in this sense); Lemma 3 and Corollary 5, for a follow-up mission that references Theorem 4; the smooth-case Hessian discussion; the infinite-horizon remarks.
  • Welcome contributions. Reusable lemmas on submodularity over sublattices of Rn\mathbb R^nRn (partial minimization, Topkis 2.7.6), on preservation of submodularity under integration, and on measurability of the value functions.

Selected references

  • P. Zipkin, On the structure of lost-sales inventory models, Operations Research 56(4) (2008) 937–944. https://doi.org/10.1287/opre.1070.0482
  • S. Karlin and H. Scarf, Inventory models of the Arrow–Harris–Marschak type with time lag, in Arrow, Karlin, Scarf (eds.), Studies in the Mathematical Theory of Inventory and Production, Stanford University Press, 1958.
  • T. E. Morton, Bounds on the solution of the lagged optimal inventory equation with no demand backlogging and proportional costs, SIAM Review 11(4) (1969) 572–596. https://doi.org/10.1137/1011090
  • K. Murota, Discrete Convex Analysis, SIAM, 2003. https://doi.org/10.1137/1.9780898718508
  • D. M. Topkis, Supermodularity and Complementarity, Princeton University Press, 1998.
8 thms1 active userReviewed
AnalysisProbabilityStochastic Systems·Captain: mikedeng1

Some Useful Functions for Functional Limit Theorems 2: Addition, Supremum and the Reflecting Barrier Preserve J₁ Convergence of Linearly Centered PathsResearch Paper

Motivation

Queueing models often represent work arriving to and leaving a server by a real-valued input path. The resulting workload cannot fall below a barrier. Functional limit theorems describe what happens to such paths under scaling, but a limit for the input path is useful only when it can be carried through the workload transformation. Ward Whitt’s 1980 paper studies several path transformations that occur in stochastic-process limits and states conditions under which their limits can be computed in Skorohod topologies. Section 6 treats the running supremum and a reflecting barrier, including paths with a linear drift that changes with the scaling index. This mission formalizes the three-regime reflection result, Theorem 6.4, and the source results on its dependency path. Whitt (1980)

The result is already proved in the paper. Its formal Lean statement remains an open proof target here. The mission is therefore concerned with formalizing the known theorem, including its exact jump conditions and convergence modes, rather than proposing a new queueing claim. Whitt (1980), pp. 80–81

Setting

A càdlàg path on a real interval TTT is right-continuous and has a limit from the left wherever TTT approaches a time from the left. The collection of such real-valued paths is D(T,R)D(T,\mathbb R)D(T,R). For a compact interval [a,b][a,b][a,b], the J1J_1J1​ distance between xxx and yyy allows an increasing homeomorphism λ\lambdaλ of time and measures the larger of its distance from the identity and the uniform distance between xxx and y∘λy\circ\lambday∘λ. On a general interval, J1J_1J1​ convergence is specified by restrictions to compact subintervals whose endpoints are continuity points of the limit or endpoints of TTT. The paper uses complete separable metric spaces for general path-space results; Section 6 specializes to real-valued paths and intervals with 000 as a closed left endpoint. Whitt (1980), pp. 70, 80

For x∈D(T,R)x\in D(T,\mathbb R)x∈D(T,R) and t∈Tt\in Tt∈T, define the running supremum x↑(t)=sup⁡0≤s≤tx(s)x^{\uparrow}(t)=\sup_{0\le s\le t}x(s)x↑(t)=sup0≤s≤t​x(s) and running infimum x↓(t)=inf⁡0≤s≤tx(s)x^{\downarrow}(t)=\inf_{0\le s\le t}x(s)x↓(t)=inf0≤s≤t​x(s). The reflecting-barrier map is f(x)(t)=x(t)−x↓(t)f(x)(t)=x(t)-x^{\downarrow}(t)f(x)(t)=x(t)−x↓(t). This is the paper’s definition: it subtracts the full running infimum. The identity path is e(t)=te(t)=te(t)=t, and θ\thetaθ denotes the zero path. A path has no positive jumps if x(t)≤x(t−)x(t)\le x(t-)x(t)≤x(t−) for each t>0t>0t>0 in TTT; no negative jumps reverses that inequality. Whitt (1980), pp. 80–81

Formalization targets

The goal is Theorem 6.4. Suppose xn−cne→xx_n-c_ne\to xxn​−cn​e→x in J1J_1J1​ and x(0)=0x(0)=0x(0)=0. Its three conclusions are

cn→c  ⟹  f(xn)→f(x+ce)(J1),cn→+∞  ⟹  f(xn)−cne→x(J1),cn→−∞ and x has no positive jumps  ⟹  f(xn)→θ(U).\begin{aligned} c_n\to c &\implies f(x_n)\to f(x+ce) &&(J_1),\\ c_n\to+\infty &\implies f(x_n)-c_ne\to x &&(J_1),\\ c_n\to-\infty\ \text{and }x\text{ has no positive jumps} &\implies f(x_n)\to\theta &&(U). \end{aligned}cn​→ccn​→+∞cn​→−∞ and x has no positive jumps​⟹f(xn​)→f(x+ce)⟹f(xn​)−cn​e→x⟹f(xn​)→θ​​(J1​),(J1​),(U).​

Here UUU means uniform convergence on every compact subinterval. The goal keeps all three drift regimes together, as the printed theorem does. Its milestones are the finite-partition fact cited from Billingsley, the interval-splitting Lemma 2.2, continuity of addition at limits without shared discontinuities in Theorem 4.1, the running-supremum inequality in Theorem 6.1, the three-regime supremum theorem 6.2, and the paper’s explicit claim that fff is J1J_1J1​-continuous. Whitt (1980), pp. 71, 79–81

Significance

The theorem specifies which output path appears after reflection as the linear centering changes. Finite drift leaves a reflected limit with drift included; large positive drift leaves the centered input limit; large negative drift drives the reflected path uniformly to zero when upward jumps of the limit are excluded. The distinction between J1J_1J1​ and the stronger compact-uniform mode in the last case is part of the result. These statements make the paper’s barrier transformation usable when an input-process limit is already known. Whitt (1980), Theorem 6.4

A completed formal development would establish reusable path-space components: a precise J1J_1J1​ convergence interface for real intervals, a theorem about addition with disjoint discontinuities, and continuity properties of running extrema and reflection. The present Lean items state these results and compile with proof placeholders. No machine-checked proof of these mission theorems is claimed here. The milestone list follows the claims Whitt states or cites, so later proofs can be attached to the same mathematical targets. Whitt (1980), §§2, 4, 6

Difficulty

A uniform estimate alone does not establish a J1J_1J1​ limit: separate paths may converge using different changes of time, and addition can fail when the limiting paths jump together. Whitt’s Theorem 4.1 gives continuity precisely at pairs with disjoint discontinuity sets. In the reflection theorem, a linearly growing drift also changes which extrema matter, and the sign of a jump decides whether the large-drift conclusion can hold. The negative-drift conclusion asks for compact-uniform convergence, which is stronger than merely retaining J1J_1J1​ convergence. These are real constraints on the statement; dropping a jump condition or replacing UUU by J1J_1J1​ changes the theorem. Whitt (1980), pp. 78–81

Formalization scope

Paths are total functions R→S\mathbb R\to SR→S, but only their values on the stated interval count. The Lean definitions require càdlàg membership of both prelimit and limit paths in each J1J_1J1​ convergence claim. A time change on [a,b][a,b][a,b] is strictly increasing, continuous, and onto; this represents the paper’s increasing homeomorphism. Uniform and J1J_1J1​ distances take values in the extended nonnegative reals, so an unbounded supremum cannot silently become zero. Continuity points use continuity within TTT, and the discontinuity set likewise contains only points of TTT. Sequential convergence is used because the paper’s J1J_1J1​ spaces are metrizable. These choices encode the paper’s path-space conventions rather than imposing conditions on values outside TTT. Whitt (1980), §§2, 4, 6

The compact form of Theorem 6.1 uses metric (2.1); the noncompact metric (2.2) is outside this mission’s definition layer. Theorem 4.1 retains the paper’s complete separable metric-space setting and metric compatibility of addition. Its measurability clause is omitted: this mission does not construct the Borel measurable space of D(T,S)D(T,S)D(T,S), and a product measurable structure on all total functions would express a different claim. The source’s f(x)=x−x↓f(x)=x-x^{\downarrow}f(x)=x−x↓ is used on both prelimit and limit paths; it is not replaced by a clipped running infimum. Whitt (1980), pp. 70, 78–81

The definition layer, the addition theorem, the running-extremum results, and the barrier continuity statement are welcome proof targets. The theorem’s centered-path hypothesis remains a genuine J1J_1J1​ convergence assertion, and the jump conditions apply to the limit path only. A vacuous convergence predicate or an independently assumed conclusion would not represent this mission’s goal.

Selected references

  • Ward Whitt, Some Useful Functions for Functional Limit Theorems, Mathematics of Operations Research 5(1), 1980, pp. 67–85. DOI: 10.1287/moor.5.1.67
9 thms1 active userReviewed
OptimizationTheoretical Computer Science·Captain: mikedeng1

A Scheduling Model for Reduced CPU Energy: For P(s) = s², the Average Rate Heuristic Has Competitive Ratio Between 4 and 8Research Paper

Why processor speed changes the scheduling problem

A processor can save energy by running slowly when work is light, but a deadline may force it to run faster. Yao, Demers, and Shenker formulated this tradeoff as an offline and online scheduling problem for a variable-speed processor in their 1995 FOCS paper. Their Average Rate Heuristic (AVR) uses only each job's arrival, deadline, and required work to set speed. This mission formalizes their quadratic-power analysis: AVR's worst-case energy is at least four and at most eight times the minimum possible energy. The authors prove these bounds in Theorem 2, rather than an exact value.

The question matters when an online scheduler must choose speed before seeing later jobs. Deadline feasibility alone does not measure energy: processing the same number of cycles at twice the speed for half as long can cost more when power grows superlinearly. The paper's model isolates that dependence in a power function P(s)P(s)P(s), so the competitive guarantee measures the energy price of AVR's speed choices. The initial motivation, model, and heuristic are in §§1–4 of the paper.

Jobs, schedules, and the average-rate rule

Fix a nondegenerate time window [t0,t1][t_0,t_1][t0​,t1​]. A finite job instance JJJ assigns each job jjj an arrival time aja_jaj​, deadline bj>ajb_j>a_jbj​>aj​, and nonnegative required work RjR_jRj​ in CPU cycles. Every job window [aj,bj][a_j,b_j][aj​,bj​] lies inside [t0,t1][t_0,t_1][t0​,t1​]. A schedule S=(s,job⁡)S=(s,\operatorname{job})S=(s,job) has nonnegative processor speed s(t)s(t)s(t) and selects at most one job at each time. Speed and job selection are constant between finitely many breakpoints. The schedule is feasible if each job receives its full work inside its own window:

∫ajbjs(t) 1{job⁡(t)=j} dt=Rjfor every j.\int_{a_j}^{b_j}s(t)\,\mathbf 1\{\operatorname{job}(t)=j\}\,dt=R_j \qquad\text{for every }j.∫aj​bj​​s(t)1{job(t)=j}dt=Rj​for every j.

For a power function PPP, its energy is EP(S)=∫t0t1P(s(t)) dtE_P(S)=\int_{t_0}^{t_1}P(s(t))\,dtEP​(S)=∫t0​t1​​P(s(t))dt. An optimal schedule minimizes this quantity among feasible schedules. These definitions follow §2, p. 375. The intensity of an interval [z,z′][z,z'][z,z′] is the total work of jobs whose full windows it contains, divided by z′−zz'-zz′−z. An interval with maximum intensity is critical; Theorem 1 identifies the processor speed on that interval in an optimal schedule §3, p. 375.

Each job's density is dj=Rj/(bj−aj)d_j=R_j/(b_j-a_j)dj​=Rj​/(bj​−aj​), extended as a step function equal to djd_jdj​ on [aj,bj][a_j,b_j][aj​,bj​] and zero elsewhere. AVR runs at speed ∑jdj(t)\sum_j d_j(t)∑j​dj​(t) and chooses the earliest-deadline available job. For quadratic power, its energy is

AVR⁡(J)=∫t0t1(∑jdj(t))2dt.\operatorname{AVR}(J)=\int_{t_0}^{t_1}\left(\sum_jd_j(t)\right)^2dt.AVR(J)=∫t0​t1​​(j∑​dj​(t))2dt.

The competitive ratio is the least upper bound of AVR⁡(J)/OPT⁡(J)\operatorname{AVR}(J)/\operatorname{OPT}(J)AVR(J)/OPT(J) over instances, where OPT⁡(J)\operatorname{OPT}(J)OPT(J) is the least feasible energy §4, p. 376, Eq. (2).

Formalization targets

The goal is the paper's Theorem 2, p. 381:

4≤r≤8(P(s)=s2).4\le r\le 8\qquad(P(s)=s^2).4≤r≤8(P(s)=s2).

The upper bound compares AVR with every feasible schedule. The lower bound says that every constant strictly below four is exceeded by the AVR-to-feasible-energy ratio on some instance. This formulation does not claim that one finite instance attains four.

The milestone path follows the paper: Theorem 1 characterizes optimal speed; Lemma 5.1 reduces to jobs of type A or B; Eq. (5) splits their costs; Lemma 5.2 and Eq. (7) control the A contribution; Lemmas 5.3–5.6 reduce the analysis to canonical instances; Lemma 5.8 supplies a matrix bound; Lemma 5.7 gives FA≤4OPT⁡AF_A\le4\operatorname{OPT}_AFA​≤4OPTA​ and FB≤4OPT⁡BF_B\le4\operatorname{OPT}_BFB​≤4OPTB​; Example 2 approaches the lower constant. These statements and their order come from §5, pp. 378–381. The paper also states a higher-power result, Theorem 3, for real p≥2p\ge2p≥2:

pp≤rp≤2p−1pp.p^p\le r_p\le 2^{p-1}p^p.pp≤rp​≤2p−1pp.

That result is a companion item. The extended abstract leaves its full proof to a longer paper §6, pp. 381–382.

What the bounds provide

The upper bound guarantees that AVR's energy is controlled by a fixed multiple of the offline optimum for every finite job set, even though AVR sets speed from individual job densities. The lower family rules out a guarantee below four for this heuristic. Together they delimit the performance of AVR under quadratic power; they do not give its exact worst-case ratio. The A/B reductions and the tree-induced matrix estimate are useful mathematical interfaces for later analyses of interval-structured schedules.

The paper proves Theorem 2 in the mathematical sense. This mission supplies machine-checkable statements and definitions, with proofs left open for solvers. Closing it requires formal proofs of the reduction steps, the matrix estimate, and the limiting example, plus an explicit connection from the optimal-schedule analysis to comparison with every feasible schedule. No completed Lean proof is claimed here.

The main difficulty

The simple calculation (∑jdj(t))2\left(\sum_j d_j(t)\right)^2(∑j​dj​(t))2 exposes overlap between job windows but does not say how that overlap compares with an optimal schedule. An optimal schedule may run jobs in a different order and may preempt them. The paper therefore changes the instance while preserving the optimal speed profile: it first separates cumulative execution into two types, then reduces to single execution intervals and specially aligned, nested windows. The matrix inequality controls the resulting expression. A pointwise comparison of AVR speed with optimal speed cannot replace these reductions, because an optimal schedule can concentrate work on critical intervals while AVR spreads each job's density across its entire window §5, pp. 377–381.

Formalization scope

Jobs use Fin n, allowing the empty instance. The instance requires t0<t1t_0<t_1t0​<t1​, aj<bja_j<b_jaj​<bj​, containment in the window, and Rj≥0R_j\ge0Rj​≥0. A schedule uses real-valued speed and an optional job label; none denotes no assigned job. Its piecewise-constant requirement is witnessed by finitely many increasing breakpoints. Values at breakpoints are unrestricted by the piecewise-constant clauses, and speed comparisons in the theorems are almost-everywhere statements. Job windows are closed, as printed; endpoints do not affect the integrals. A schedule may have positive speed while no job is assigned, since the source does not equate idleness with zero speed.

Every energy expression is a real interval integral. The schedule conditions ensure the integrands in theorem hypotheses are piecewise constant away from finitely many points. Density denominators are positive. An execution-speed quotient for a zero-work job can have zero denominator, in which case Lean returns zero; such a job contributes no executed work. The goal uses quantified comparisons with feasible schedules rather than a real infimum, and Lemma 5.6 uses all upper bounds on a nonempty canonical family rather than a real supremum. The lower bounds quantify over every smaller constant, avoiding a false attainment claim. These choices guard against default values for empty or unbounded extrema.

Theorem 1's printed convexity hypothesis is strengthened to strict convexity and nondecreasing power on nonnegative speeds: its stated speed uniqueness fails for a linear power function, and a decreasing power function rewards idle speed. This repair is explicit in that item's description. Existence of a quadratic optimum is included as a milestone from the algorithm of §3, although it is not a separately numbered theorem. Example 2 includes the variable exponent e≥1e\ge1e≥1, the limiting ratios at e=1e=1e=1 and e=3/2e=3/2e=3/2, and an eventual upper bound of four across the family. The A-job order resolves exact ties by index. For canonical and tree-induced nesting, intervals that meet only at an endpoint count as disjoint; this admits the paper's displayed four-interval matrix.

The development needs Mathlib's real interval integrals, finite sums, measures of intervals, matrix eigenvalues, and real-power limits. The finite schedule model, canonical-instance data, and tree-induced matrix interface can be reused beyond this particular ratio bound. Contributions may establish these interfaces' elementary properties, the paper's milestone theorems, or independent proofs of the goal, provided they preserve the stated job and schedule classes.

Selected references

  • Frances Yao, Alan Demers, and Scott Shenker, A Scheduling Model for Reduced CPU Energy, Proceedings of the 36th IEEE Symposium on Foundations of Computer Science, 1995, pp. 374–382. DOI 10.1109/SFCS.1995.492493.
19 thms1 active userReviewed
AnalysisProbabilityStochastic Systems·Captain: mikedeng1

Some Useful Functions for Functional Limit Theorems 4: Time Reversal Is a J₁ Isometry and Reversal from the Origin Is 2-LipschitzResearch Paper

Why time reversal matters

Functional limit theorems describe the behavior of entire time-indexed paths under scaling. In queueing and other stochastic models, a path of departures, workload, or accumulated input may be related to another path by reversing a finite observation window. To carry a limit theorem through that change of viewpoint, one needs to know how reversal acts on the topology of the path space. Ward Whitt's 1980 paper studies composition, reflection, first passage, and time reversal as maps that transfer functional limit theorems to derived processes (Whitt 1980).

The time-reversal result is a precise metric statement. Ordinary reversal preserves the Skorohod J1J_1J1​ distance exactly. A version shifted to begin at the group identity satisfies a bound with constant two. These facts give continuity and, when the random paths satisfy the hypotheses, make reversal available to the continuous mapping theorem. Whitt's introduction explains how deterministic path continuity connects to stochastic-process limits (Whitt 1980, pp. 67–69).

Paths, distances, and reversal

Let SSS be a complete separable metric group, written additively, with identity 000 and metric mmm. The metric is translation invariant: moving both points by the same group element leaves their distance unchanged. Let DDD contain the càdlàg paths x:[0,1]→Sx:[0,1]\to Sx:[0,1]→S, meaning paths that are right continuous and possess left limits. Section 8 additionally requires x(1)=x(1−)x(1)=x(1-)x(1)=x(1−), so the terminal value agrees with the value approached from earlier times. Let Dθ={x∈D:x(0)=0}D_\theta=\{x\in D:x(0)=0\}Dθ​={x∈D:x(0)=0}. These are the spaces Whitt fixes for time reversal (Whitt 1980, p. 83).

The uniform distance is ρ(x,y)=sup⁡0≤t≤1m(x(t),y(t))\rho(x,y)=\sup_{0\le t\le1}m(x(t),y(t))ρ(x,y)=sup0≤t≤1​m(x(t),y(t)). A time change λ∈Λ\lambda\in\Lambdaλ∈Λ is an increasing homeomorphism of [0,1][0,1][0,1] onto itself; e(t)=te(t)=te(t)=t is the identity time change. The J1J_1J1​ distance used in the paper is

d(x,y)=inf⁡λ∈Λmax⁡{ρ(λ,e),ρ(x,y∘λ)}.d(x,y)=\inf_{\lambda\in\Lambda} \max\{\rho(\lambda,e),\rho(x,y\circ\lambda)\}.d(x,y)=λ∈Λinf​max{ρ(λ,e),ρ(x,y∘λ)}.

Two paths may be close when their values can be matched after a small distortion of time. This is the paper's equation (2.1), rather than an unspecified equivalent J1J_1J1​ metric (Whitt 1980, p. 70).

The reverse-time map reads the left limit at the reflected time:

R(x)(t)={x((1−t)−),0≤t<1,x(0),t=1,r(x)(t)=R(x)(t)−x(1).R(x)(t)= \begin{cases} x((1-t)-),&0\le t<1,\\ x(0),&t=1, \end{cases} \qquad r(x)(t)=R(x)(t)-x(1).R(x)(t)={x((1−t)−),x(0),​0≤t<1,t=1,​r(x)(t)=R(x)(t)−x(1).

The left limit is essential: the direct value x(1−t)x(1-t)x(1−t) places jumps on the wrong side and generally does not give a right-continuous reverse path. The terminal convention gives R(x)(0)=x(1)R(x)(0)=x(1)R(x)(0)=x(1), so r(x)(0)=0r(x)(0)=0r(x)(0)=0. Whitt also uses the reflected time change (−r)(λ)(t)=1−λ(1−t)(-r)(\lambda)(t)=1-\lambda(1-t)(−r)(λ)(t)=1−λ(1−t) in the proof (Whitt 1980, pp. 83–84).

Formalization targets

The goal is Whitt's Theorem 8.1. For every x,y∈Dx,y\in Dx,y∈D,

d(Rx,Ry)=d(x,y),d(rx,ry)≤2d(x,y).d(Rx,Ry)=d(x,y),\qquad d(rx,ry)\le 2d(x,y).d(Rx,Ry)=d(x,y),d(rx,ry)≤2d(x,y).

The second inequality also applies when both paths lie in DθD_\thetaDθ​, since Dθ⊆DD_\theta\subseteq DDθ​⊆D. The constant is the paper's printed constant, not a parameter to optimize in this mission (Whitt 1980, Theorem 8.1, p. 83).

The milestones record four claims from the theorem's proof: reversal preserves its path spaces and has inverse behavior on its natural domains; reflected time changes stay in Λ\LambdaΛ; reversal obeys exact and bounded uniform-distance relations; and reversal interacts with time changes through (−r)(λ)(-r)(\lambda)(−r)(λ). Their descriptions reproduce the printed proof sentences. The proof calls the time-change cost a “minimum” once, although equation (2.1) and the displayed calculation use a maximum. The formal target follows equation (2.1) (Whitt 1980, pp. 70, 84).

What the result gives

An isometry preserves every J1J_1J1​ distance, so ordinary reversal preserves convergent and Cauchy sequences in this metric. The bound for rrr makes shifted reversal Lipschitz and therefore continuous. A functional limit theorem for input paths can then be translated into one for the corresponding reversed paths whenever the path-space hypotheses hold. The result is already proved in Whitt's paper; the open work here is a machine-checked development of its definitions, supporting claims, and metric theorem.

Formalizing the result also creates reusable interfaces for endpoint-constrained càdlàg paths, left-limit reversal, uniform path distance, and reflected time changes. Those interfaces may support later statements about reversed stochastic models. This mission does not assert weak convergence of probability laws or Borel measurability; those would require a separate measurable-space development for DDD.

Mathematical difficulty

The immediate idea t↦x(1−t)t\mapsto x(1-t)t↦x(1−t) does not preserve càdlàg paths at jumps: a right-continuous jump becomes left-continuous. The correction uses left limits, but endpoint behavior then matters. Without x(1)=x(1−)x(1)=x(1-)x(1)=x(1−), the shifted reverse path need not start at the identity. The J1J_1J1​ assertion also concerns an infimum over all allowed time changes, so a uniform-distance statement alone does not establish the theorem. These are the points at which reflection of the time axis becomes a claim about the precise path-space topology (Whitt 1980, pp. 83–84).

Formalization scope

Lean represents paths as total functions R→S\mathbb R\to SR→S; all path conditions, metrics, and equality claims read only [0,1][0,1][0,1]. Càdlàg membership requires right continuity and left limits within the interval, with x(1)=x(1−)x(1)=x(1-)x(1)=x(1−) included in DDD. Completeness and separability are explicit because the paper calls SSS a complete separable metric space. The additive metric group is represented by an additive commutative group with an explicit translation-invariance hypothesis. This chooses the commutative reading exemplified by S=RkS=\mathbb R^kS=Rk; the paper uses additive notation but does not separately state commutativity.

The paper's Λ\LambdaΛ is encoded by strict increase, continuity, and surjectivity on [0,1][0,1][0,1], which characterize increasing homeomorphisms there. The paper's ddd is equation (2.1) on both sides of the goal. Lean uses extended nonnegative distances for ρ\rhoρ and ddd, avoiding the default value a real supremum can assign to an empty or unbounded set. The claims do not become trivial by dropping DDD membership: it is an explicit hypothesis, and the zero path over R\mathbb RR satisfies it. A local, sorry-free check confirms that instance. Two easier statements are excluded on purpose: the goal is stated for the J1J_1J1​ distance ddd, not the uniform distance ρ\rhoρ, and RRR reads the left limit x((1−t)−)x((1-t)-)x((1−t)−), not the value x(1−t)x(1-t)x(1−t).

The proof's first sentence says the maps are one-to-one and onto. The map r:D→Dθr:D\to D_\thetar:D→Dθ​ forgets an additive offset and is not injective on all of DDD; the formal milestone expresses involution for RRR on DDD and for the restriction of rrr to DθD_\thetaDθ​. One identity printed for rrr with x∈Dθx\in D_\thetax∈Dθ​ also holds for every x∈Dx\in Dx∈D; the formal milestone uses that broader domain and says so in its item description. Contributions are welcome for the left-limit and time-change infrastructure as well as the four listed statements and the goal.

Selected references

  • Ward Whitt, Some Useful Functions for Functional Limit Theorems, Mathematics of Operations Research 5(1), 1980, pp. 67–85. DOI: 10.1287/moor.5.1.67.
7 thms1 active userReviewed
OptimizationProbability·Captain: mikedeng1

An Airline Seat Management Model for a Single Leg Route When Lower Fare Classes Book First: The Marginal Seat Value ΔZ_m(n) Decreases in n, So Rejecting Class m Exactly When n ≤ k_m Is OptimalResearch Paper

Motivation

An airline sells the seats of one flight leg at several fares. Cheap fares are usually booked well before departure and expensive ones close to it, so every early request poses the same question: is the certain revenue of selling a seat now worth more than the chance of selling it later at a higher fare? The policy that answers it is seat inventory control, a core problem of airline revenue management.

For two fare classes the answer is Littlewood's rule (1972): accept a low-fare request as long as the low fare is at least the high fare times the probability that high-fare demand would fill the remaining seats. Belobaba (1987) extended it heuristically to several classes through expected marginal seat revenue (EMSR) protection levels, which are not optimal in general. Wollmer (Operations Research 40(1), 1992) solved the multi-class problem exactly under the assumption that the classes book in order of increasing fare: he showed that an optimal policy is described by one critical value per fare class, computed by a short recursion, and accepts a request precisely when the number of empty seats exceeds the critical value of its class. Brumelle and McGill (Operations Research 41(1), 1993) and later work on nested booking limits reached optimality of static protection levels by other routes.

Setting

There are c≥2c \ge 2c≥2 fare classes, numbered 1,…,c1, \dots, c1,…,c from the highest fare to the lowest, with fares

r1>r2>⋯>rc>0.r_1 > r_2 > \cdots > r_c > 0 .r1​>r2​>⋯>rc​>0.

Class mmm has a random demand Dm∈{0,1,2,… }D_m \in \{0, 1, 2, \dots\}Dm​∈{0,1,2,…}, the number of its future booking requests, with probabilities P[Dm=i]P[D_m = i]P[Dm​=i], P[Dm≥n]P[D_m \ge n]P[Dm​≥n], P[Dm≤i]P[D_m \le i]P[Dm​≤i]. Seats form one pool, and nnn denotes the number of empty seats. Lower classes book first: all requests of class mmm arrive before those of classes 1,…,m−11, \dots, m-11,…,m−1. The paper assumes r2<r1P[D1≥1]r_2 < r_1 P[D_1 \ge 1]r2​<r1​P[D1​≥1], so that a class 2 request is refused when one seat remains.

The optimal value Zm(n)Z_m(n)Zm​(n) is the expected revenue under an optimal policy when nnn seats are empty and only classes 1,…,m1, \dots, m1,…,m may still book. It satisfies Z0≡0Z_0 \equiv 0Z0​≡0 and

Zm(n)=E[max⁡0≤x≤min⁡(Dm,n)(rmx+Zm−1(n−x))],Z_m(n) = \mathbb E\Big[\max_{0 \le x \le \min(D_m, n)} \big(r_m x + Z_{m-1}(n-x)\big)\Big],Zm​(n)=E[0≤x≤min(Dm​,n)max​(rm​x+Zm−1​(n−x))],

where xxx is the number of class-mmm requests accepted. The marginal seat value is ΔZm(n)=Zm(n)−Zm(n−1)\Delta Z_m(n) = Z_m(n) - Z_m(n-1)ΔZm​(n)=Zm​(n)−Zm​(n−1) for n≥1n \ge 1n≥1, and the critical values are

k1=0,km=max⁡{ n≥1∣rm<ΔZm−1(n) }(m>1).(2)k_1 = 0, \qquad k_m = \max\{\, n \ge 1 \mid r_m < \Delta Z_{m-1}(n) \,\} \quad (m > 1). \tag{2}k1​=0,km​=max{n≥1∣rm​<ΔZm−1​(n)}(m>1).(2)

The critical-value rule for class mmm rejects a class-mmm request when n≤kmn \le k_mn≤km​ and accepts it when n≥km+1n \ge k_m + 1n≥km​+1; it accepts min⁡(Dm,(n−km)+)\min(D_m, (n-k_m)^+)min(Dm​,(n−km​)+) of the class's requests.

Formalization targets

Goal: Theorem 2 with the rejection rule of §3

For every class 1≤m≤c1 \le m \le c1≤m≤c, ΔZm(n)\Delta Z_m(n)ΔZm​(n) is nonincreasing in n≥1n \ge 1n≥1. For every class 2≤m≤c2 \le m \le c2≤m≤c, the critical value kmk_mkm​ exists, Zm(n)=Zm−1(n)Z_m(n) = Z_{m-1}(n)Zm​(n)=Zm−1​(n) for n≤kmn \le k_mn≤km​, the critical-value rule attains the maximum in the recursion for Zm(n)Z_m(n)Zm​(n) (and accepting more requests is strictly worse), and for every j≥1j \ge 1j≥1

Zm(km+j)=Zm(km)+∑i=0j−1{ΔZm−1(km+j−i) P[Dm≤i]+rm P[Dm≥i+1]}.(6)Z_m(k_m+j) = Z_m(k_m) + \sum_{i=0}^{j-1} \Big\{ \Delta Z_{m-1}(k_m+j-i)\,P[D_m \le i] + r_m\,P[D_m \ge i+1] \Big\}. \tag{6}Zm​(km​+j)=Zm​(km​)+i=0∑j−1​{ΔZm−1​(km​+j−i)P[Dm​≤i]+rm​P[Dm​≥i+1]}.(6)

Milestones

  1. Eq. (3): Z1(n)=r1{nP[D1≥n]+∑j=0n−1jP[D1=j]}Z_1(n) = r_1\{nP[D_1 \ge n] + \sum_{j=0}^{n-1} jP[D_1 = j]\}Z1​(n)=r1​{nP[D1​≥n]+∑j=0n−1​jP[D1​=j]}.
  2. Eq. (4): ΔZ1(n)=r1P[D1≥n]\Delta Z_1(n) = r_1 P[D_1 \ge n]ΔZ1​(n)=r1​P[D1​≥n], nonincreasing in nnn.
  3. The note after (2b): if ΔZm−1\Delta Z_{m-1}ΔZm−1​ is nonincreasing, then rm<ΔZm−1(n)r_m < \Delta Z_{m-1}(n)rm​<ΔZm−1​(n) for n≤kmn \le k_mn≤km​ and rm≥ΔZm−1(n)r_m \ge \Delta Z_{m-1}(n)rm​≥ΔZm−1​(n) for n≥km+1n \ge k_m + 1n≥km​+1.
  4. The rule of §3: if ΔZm−1\Delta Z_{m-1}ΔZm−1​ is nonincreasing, rejecting class mmm exactly when n≤kmn \le k_mn≤km​ is optimal, and Zm=Zm−1Z_m = Z_{m-1}Zm​=Zm−1​ on n≤kmn \le k_mn≤km​.
  5. Lemma 1: the case j=1j = 1j=1 of (6) for class mˉ+1\bar m + 1mˉ+1, and ΔZmˉ+1(kmˉ+1+1)<ΔZmˉ+1(kmˉ+1)\Delta Z_{\bar m+1}(k_{\bar m+1}+1) < \Delta Z_{\bar m+1}(k_{\bar m+1})ΔZmˉ+1​(kmˉ+1​+1)<ΔZmˉ+1​(kmˉ+1​).
  6. The marginal display in the proof of Theorem 1: ΔZmˉ+1(kmˉ+1+j)=∑i<jΔZmˉ(kmˉ+1+j−i)P[Dmˉ+1=i]+rmˉ+1P[Dmˉ+1≥j]\Delta Z_{\bar m+1}(k_{\bar m+1}+j) = \sum_{i<j} \Delta Z_{\bar m}(k_{\bar m+1}+j-i)P[D_{\bar m+1} = i] + r_{\bar m+1}P[D_{\bar m+1} \ge j]ΔZmˉ+1​(kmˉ+1​+j)=∑i<j​ΔZmˉ​(kmˉ+1​+j−i)P[Dmˉ+1​=i]+rmˉ+1​P[Dmˉ+1​≥j].
  7. Theorem 1: (6) for class mˉ+1\bar m + 1mˉ+1, and ΔZmˉ+1\Delta Z_{\bar m+1}ΔZmˉ+1​ nonincreasing above kmˉ+1k_{\bar m+1}kmˉ+1​.

Significance

The theorem reduces a dynamic program over all booking policies to one integer per fare class. The critical values are computed class by class from (2b) and (6), in a number of operations polynomial in the number of seats and classes, and the resulting policy is a nested booking limit that reservation systems can implement directly. The abstract states two further properties derived from it: the critical value decreases with the fare and is zero for the highest class.

The result is proved in the paper; it has no machine-checked proof. A complete formalization would give a verified account of the multi-class seat allocation problem with sequential booking, including the existence of the critical values, which the paper presupposes. The intermediate statements (the concavity-type property of ZmZ_mZm​ and the closed form (6)) are reusable for nested-booking-limit results in related models.

Difficulty

The induction of the paper is short, but it relies on one property: ΔZm−1\Delta Z_{m-1}ΔZm−1​ must be nonincreasing for the threshold rule to be optimal for class mmm, and the threshold rule must be optimal for ΔZm\Delta Z_mΔZm​ to inherit the property. These two facts have to be carried together through the induction. The step at the threshold itself (n=km+1n = k_m + 1n=km​+1) behaves differently from the rest: there the marginal value drops strictly, and the argument above the threshold needs the maximality in (2b), not monotonicity alone. The existence of kmk_mkm​ is not argued on the page: it requires bounding the marginal seat value away from rmr_mrm​ for large nnn without assuming that demands have finite mean. The proofs of Lemma 1 and Theorem 1 also contain index slips (kmˉk_{\bar m}kmˉ​ printed for kmˉ+1k_{\bar m+1}kmˉ+1​) that a formal proof must not reproduce.

Formalization scope

All statements live in the namespace LowerFareFirst.Critical and rest on one definition module, LowerFareFirst.Critical.Model. Classes are natural numbers starting at 111; fares are a sequence r:N→Rr : \mathbb N \to \mathbb Rr:N→R and demands a sequence of measurable maps D:N→Ω→ND : \mathbb N \to \Omega \to \mathbb ND:N→Ω→N on a probability space (Ω,μ)(\Omega, \mu)(Ω,μ), of which only the entries 1,…,c1, \dots, c1,…,c are constrained. Probabilities are μ.real of events.

Committed conventions:

  • ZmZ_mZm​ is defined by the dynamic-programming recursion above, which is the reading of the paper's "expected revenue under an optimal policy". It is not the value of the critical-value policy and not defined by (6); either choice would make the goal definitional. Optimality of the rule is stated as attaining the maximum of the recursion for every realized demand.
  • "Decreasing" means nonincreasing. The strict reading is false: ΔZ1(n)=r1P[D1≥n]\Delta Z_1(n) = r_1 P[D_1 \ge n]ΔZ1​(n)=r1​P[D1​≥n] is constant wherever P[D1=n]=0P[D_1 = n] = 0P[D1​=n]=0.
  • kmk_mkm​ is the greatest element of {n≥1∣rm<ΔZm−1(n)}\{n \ge 1 \mid r_m < \Delta Z_{m-1}(n)\}{n≥1∣rm​<ΔZm−1​(n)}. Its existence is a conclusion of the goal, never a hypothesis; Lemma 1, Theorem 1 and the intermediate milestones take it as a hypothesis, as the page does.
  • The standing assumptions of §1 are bundled in one predicate: probability measure, measurable demands, c≥2c \ge 2c≥2, strictly decreasing fares, r2<r1P[D1≥1]r_2 < r_1 P[D_1 \ge 1]r2​<r1​P[D1​≥1], and rc>0r_c > 0rc​>0, an addition to the page: with rc≤0r_c \le 0rc​≤0 and unbounded demand the maximum in (2b) need not exist.
  • Independence of the demands is not assumed: the recursion uses only their marginal laws. For independent demands, the paper's setting, ZmZ_mZm​ is the optimal expected revenue.
  • Where the proofs print kmˉk_{\bar m}kmˉ​ for kmˉ+1k_{\bar m+1}kmˉ+1​, and where §3 says "class m+1m + 1m+1" for class mmm, the statements use the corrected index.

A complete development needs elementary facts about expectations of functions of an integer-valued random variable and finite maximization; nothing beyond Mathlib's measure theory. Proofs of any milestone are welcome, as are proofs of the abstract's remarks (kmk_mkm​ nonincreasing in the fare) as separate theorems.

Selected references

  • R. D. Wollmer, An Airline Seat Management Model for a Single Leg Route When Lower Fare Classes Book First, Operations Research 40(1), 26–37, 1992. https://doi.org/10.1287/opre.40.1.26
  • K. Littlewood, Forecasting and Control of Passenger Bookings, AGIFORS Symposium Proceedings 12, 1972; reprinted in Journal of Revenue and Pricing Management 4(2), 111–123, 2005. https://doi.org/10.1057/palgrave.rpm.5170134
  • P. P. Belobaba, Airline Yield Management: An Overview of Seat Inventory Control, Transportation Science 21(2), 63–73, 1987. https://doi.org/10.1287/trsc.21.2.63
  • S. L. Brumelle, J. I. McGill, Airline Seat Allocation with Multiple Nested Fare Classes, Operations Research 41(1), 127–137, 1993. https://doi.org/10.1287/opre.41.1.127
9 thms1 active userReviewed
Dynamic ProgrammingMarkov Chain·Captain: mikedeng1

Discrete Dynamic Programming with Sensitive Discount Optimality Criteria 2: No Improvement of Order n+1 Implies n-Discount Optimality, Which Rules Out Improvement of Order nResearch Paper

Motivation

A finite Markov decision process is usually solved under one of two criteria: the expected total discounted reward, for a fixed interest rate, or the long-run average reward. Neither is satisfactory when the interest rate is small and not known precisely. Blackwell (Blackwell 1962) showed that some stationary policy is optimal for every discount factor close enough to one, and that the average-reward criterion cannot tell apart policies with the same gain but very different transient rewards. Veinott's 1969 paper interpolates between these two: it introduces a hierarchy of nnn-discount optimality criteria, indexed by n=−1,0,1,…n=-1,0,1,\dotsn=−1,0,1,…, each more selective than the previous, and characterizes the stationary policies that meet each level through the Laurent expansion of the discounted return in the interest rate.

Timeline. Blackwell (1962) introduced the expansion of the discounted return near β=1\beta=1β=1 to first order and the criteria now called −1+-1^+−1+ and ∞+\infty^+∞+ discount optimality. Veinott (1966) gave a policy improvement method for the bias (0+0^+0+) criterion. Miller and Veinott (1969) gave the full Laurent expansion and a policy improvement algorithm for Blackwell optimality in the stochastic case. The present paper (1969) extends this to substochastic transition matrices and to negative interest rates in the transient case, defines the n±n^\pmn± criteria, and proves Theorem 4, the subject of this mission.

Setting

There are finitely many states sss (the paper's 1,…,S1,\dots,S1,…,S); in each state a finite nonempty action set AsA_sAs​. Action aaa in state sss earns r(s,a)∈Rr(s,a)\in\mathbb Rr(s,a)∈R and leads to state ttt with weight p(t∣s,a)≥0p(t\mid s,a)\ge0p(t∣s,a)≥0, where ∑tp(t∣s,a)≤1\sum_t p(t\mid s,a)\le1∑t​p(t∣s,a)≤1 (the remaining mass is the probability of stopping). A decision rule f∈F=×sAsf\in F=\times_s A_sf∈F=×s​As​ picks one action per state; r(f)r(f)r(f) and P(f)P(f)P(f) are its reward vector and substochastic transition matrix, Q(f)=P(f)−IQ(f)=P(f)-IQ(f)=P(f)−I. A policy is a sequence π=(f1,f2,… )\pi=(f_1,f_2,\dots)π=(f1​,f2​,…) of decision rules, f∞=(f,f,… )f^\infty=(f,f,\dots)f∞=(f,f,…) is a stationary policy, and PN(π)=P(f1)⋯P(fN)P^N(\pi)=P(f_1)\cdots P(f_N)PN(π)=P(f1​)⋯P(fN​).

For an interest rate ρ>−1\rho>-1ρ>−1 and β=(1+ρ)−1\beta=(1+\rho)^{-1}β=(1+ρ)−1, the discounted return is

Vρ(π)=∑N=1∞βNPN−1(π) r(fN).V_\rho(\pi)=\sum_{N=1}^{\infty}\beta^N P^{N-1}(\pi)\,r(f_N).Vρ​(π)=N=1∑∞​βNPN−1(π)r(fN​).

For a substochastic PPP, P∗P^*P∗ is the Cesàro limit of its powers and H=(I−P+P∗)−1−P∗H=(I-P+P^*)^{-1}-P^*H=(I−P+P∗)−1−P∗ its deviation matrix.

A policy π∗\pi^*π∗ is n±n^\pmn± discount optimal if, componentwise,

lim inf⁡ρ→0±∣ρ∣−n [Vρ(π∗)−Vρ(π)]≥0for all policies π.\liminf_{\rho\to0\pm}|\rho|^{-n}\,[V_\rho(\pi^*)-V_\rho(\pi)]\ge0\quad\text{for all policies }\pi .ρ→0±liminf​∣ρ∣−n[Vρ​(π∗)−Vρ​(π)]≥0for all policies π.

Dn±D_n^\pmDn±​ is the set of fff with f∞f^\inftyf∞ n±n^\pmn± discount optimal, and D−2=FD_{-2}=FD−2​=F. The sign −-− (interest rate approaching zero from below) is considered only in the transient case, where ∑NP(f)N\sum_N P(f)^N∑N​P(f)N converges for every fff.

The Laurent coefficients of Vρ(f∞)V_\rho(f^\infty)Vρ​(f∞) are y−1±(f)=±P∗(f)r(f)y_{-1}^\pm(f)=\pm P^*(f)r(f)y−1±​(f)=±P∗(f)r(f) and yn±(f)=(∓1)nH(f)n+1r(f)y_n^\pm(f)=(\mp1)^nH(f)^{n+1}r(f)yn±​(f)=(∓1)nH(f)n+1r(f), n≥0n\ge0n≥0; with y−2±=0y_{-2}^\pm=0y−2±​=0 and r0=rr_0=rr0​=r, rn=0r_n=0rn​=0 otherwise, the test quantities are

ψn±(g,f)=rn(g)+Q(g) yn±(f)∓yn−1±(f),n≥−1.\psi_n^\pm(g,f)=r_n(g)+Q(g)\,y_n^\pm(f)\mp y_{n-1}^\pm(f),\qquad n\ge-1 .ψn±​(g,f)=rn​(g)+Q(g)yn±​(f)∓yn−1±​(f),n≥−1.

Ψn±(g,f)\Psi_n^\pm(g,f)Ψn±​(g,f) is the matrix with columns ψ−1±,…,ψn±\psi_{-1}^\pm,\dots,\psi_n^\pmψ−1±​,…,ψn±​, and Gn±(f)={g∈F:Ψn±(g,f)≻0}G_n^\pm(f)=\{g\in F:\Psi_n^\pm(g,f)\succ0\}Gn±​(f)={g∈F:Ψn±​(g,f)≻0}, where C≻0C\succ0C≻0 means that the first nonzero entry of every row of CCC is positive and C≠0C\ne0C=0.

Formalization targets

Goal: Theorem 4

For f∈Ff\in Ff∈F and n=−2,−1,…,S−1n=-2,-1,\dots,S-1n=−2,−1,…,S−1:

Gn+1±(f)=∅ ⟹ f∈Dn±,f∈Dn± ⟹ Gn±(f)=∅.G_{n+1}^\pm(f)=\varnothing\ \Longrightarrow\ f\in D_n^\pm,\qquad f\in D_n^\pm\ \Longrightarrow\ G_n^\pm(f)=\varnothing .Gn+1±​(f)=∅ ⟹ f∈Dn±​,f∈Dn±​ ⟹ Gn±​(f)=∅.

The two halves bracket Dn±D_n^\pmDn±​ between two finitely checkable conditions.

Milestones

In the order the proof uses them: the Cesàro limit and (15); Lemma 5 (the reduced resolvent); Theorem 2 (the Laurent expansion Rρ(Q)=ρ−1P∗+∑n≥0(−ρ)nHn+1R_\rho(Q)=\rho^{-1}P^*+\sum_{n\ge0}(-\rho)^nH^{n+1}Rρ​(Q)=ρ−1P∗+∑n≥0​(−ρ)nHn+1); Lemma 7 (P∗+ρHP^*+\rho HP∗+ρH nonnegative, positive diagonal, nonsingular for small ρ>0\rho>0ρ>0); Theorem 3 (the expansion of Vρ(f∞)V_\rho(f^\infty)Vρ​(f∞)); (30)–(31) (the comparison identity); Lemma 8 (the expansion of the test quantity); Lemma 9 (the coefficient identity); the nonemptiness of D∞±D_\infty^\pmD∞±​; the characterization Dn±={f:Yn±(f)⪰Yn±(g) ∀g}D_n^\pm=\{f: Y_n^\pm(f)\succeq Y_n^\pm(g)\ \forall g\}Dn±​={f:Yn±​(f)⪰Yn±​(g) ∀g}; Theorem 5 (lexicographic improvement).

Significance

Theorem 4 turns a criterion stated through limits over all, possibly non-stationary, policies into a finite test on one-step switches. It is what lets the policy improvement method of Miller and Veinott stop early when only an n±n^\pmn± discount optimal policy is sought, rather than an S±S^\pmS± (Blackwell) optimal one, and it recovers Blackwell's algorithm for −1+-1^+−1+ and Veinott's 1966 algorithm for 0+0^+0+ as the cases n=−1,0n=-1,0n=−1,0. The nnn-discount hierarchy is the standard framework for sensitive optimality in Markov decision processes and underlies the average-overtaking and bias criteria used in later work.

The results are proved in the paper (Theorem 3 and Lemma 8 by reference to Miller–Veinott). None of them is formalized: the platform has the Cesàro limit matrix and the deviation matrix as definitions (from Blackwell 1962), and an open statement of their basic properties for stochastic matrices only. This mission adds the substochastic matrix theory of §3, the Laurent expansion of discounted returns with Veinott's normalization, and the lexicographic characterization of n±n^\pmn± discount optimality.

Difficulty

The definition of Dn±D_n^\pmDn±​ compares f∞f^\inftyf∞ with every policy, and the comparison is a liminf of a scaled difference of infinite series. The obvious route, comparing Laurent coefficients, applies only to stationary policies, whose returns have Laurent expansions; reducing the comparison with arbitrary policies to stationary ones requires the existence of an ∞±\infty^\pm∞± discount optimal stationary policy, which is a separate existence theorem. The second difficulty is that P∗P^*P∗ may be singular or zero and the chain may have several recurrent classes: Lemma 7 is what replaces an analysis of the chain structure in the inductive step of the proof.

Formalization scope

Lean 4 with Mathlib, namespace VeinottSensitiveDP.Sensitive. States form a nonempty Fintype; each state has its own finite nonempty action type A s; policies are sequences ℕ → F with π 0 the paper's f1f_1f1​. All quantities are real. P∗P^*P∗ and HHH are the published limitMatrix and deviationMatrix; their properties for substochastic matrices are milestones, not assumptions. VρV_\rhoVρ​ is a tsum with Veinott's extra factor β\betaβ; every theorem about it carries the hypotheses under which the series converges. The spectral radius is that of the complexified matrix. The sign ±\pm± is a real parameter σ∈{1,−1}\sigma\in\{1,-1\}σ∈{1,−1}, and statements for σ=−1\sigma=-1σ=−1 assume the transient case. The liminf in (27) is encoded as "for every ε>0\varepsilon>0ε>0, eventually ≥−ε\ge-\varepsilon≥−ε", which is the extended-real liminf. Matrices with columns −1,…,n-1,\dots,n−1,…,n are functions of an integer column index read on [−1,n][-1,n][−1,n].

A trivializing formalization would define Dn±D_n^\pmDn±​ by comparison with stationary policies only, or directly as the lexicographic maximizers of Yn±Y_n^\pmYn±​; here Dn±D_n^\pmDn±​ is defined by (27) against all policies, and the lexicographic characterization is a milestone.

Contributions welcome: proofs of the matrix results of §3 (reusable for any work on deviation matrices and Laurent expansions of Markov chains), of the Laurent expansions, and of the existence of ∞±\infty^\pm∞± discount optimal stationary policies.

Selected references

  • A. F. Veinott, Jr., Discrete dynamic programming with sensitive discount optimality criteria, Ann. Math. Statist. 40(5):1635–1660, 1969. https://doi.org/10.1214/aoms/1177697379
  • D. Blackwell, Discrete dynamic programming, Ann. Math. Statist. 33(2):719–726, 1962. https://doi.org/10.1214/aoms/1177704593
  • A. F. Veinott, Jr., On finding optimal policies in discrete dynamic programming with no discounting, Ann. Math. Statist. 37(5):1284–1294, 1966. https://doi.org/10.1214/aoms/1177699272
  • B. L. Miller and A. F. Veinott, Jr., Discrete dynamic programming with a small interest rate, Ann. Math. Statist. 40(2):366–370, 1969. https://doi.org/10.1214/aoms/1177697700
17 thms1 active userReviewed
Convex OptimizationNumerical AnalysisOptimization·Captain: mikedeng1

A Coordinate Gradient Descent Method for Nonsmooth Separable Minimization 1: With the Armijo Rule and the Generalized Gauss–Seidel Rule, Every Cluster Point of the CGD Iterates Is StationaryResearch Paper

Motivation

Many optimization models combine a smooth loss with a convex penalty or constraint. The penalty can be nonsmooth: an absolute-value penalty encourages sparse solutions, while an indicator function enforces a feasible set. A coordinate gradient descent (CGD) method updates selected coordinates by minimizing a quadratic model and then applies a line search. Tseng and Yun studied this method for an extended-valued convex penalty, including penalties that separate by blocks of coordinates. Their global convergence result addresses a basic question for such an iteration: if its iterates have an accumulation point, must that point satisfy the first-order stationarity condition? The result and examples appear in Tseng and Yun (2009).

Coordinate updates are appealing when changing a small block is cheaper than changing the whole vector. Their convergence is delicate when the penalty is nonsmooth: a direction can be useful on one block without revealing whether the full point is stationary. The paper's Theorem 1(e) establishes a guarantee for a generalized Gauss–Seidel selection rule and a block-separable penalty. The authors note that this rules out cycling on Powell's coordinate-minimization example under their stated conditions Tseng and Yun (2009), p. 404.

Setting

Let x∈Rnx\in\mathbb R^nx∈Rn, c>0c>0c>0, and let P:Rn→(−∞,+∞]P:\mathbb R^n\to(-\infty,+\infty]P:Rn→(−∞,+∞] be proper, convex and lower semicontinuous. Its effective domain D=dom⁡PD=\operatorname{dom}PD=domP contains precisely the points where PPP is finite. Let fff be continuously differentiable on an open set containing DDD. The objective is Fc(x)=f(x)+cP(x)F_c(x)=f(x)+cP(x)Fc​(x)=f(x)+cP(x), with value +∞+\infty+∞ outside DDD. No convexity of fff is assumed.

At iteration kkk, choose a nonempty set of coordinates Jk\mathcal J^kJk and a positive-definite symmetric matrix HkH^kHk. The search direction dkd^kdk minimizes

∇f(xk)⊤d+12d⊤Hkd+cP(xk+d)subject to dj=0 for j∉Jk.\nabla f(x^k)^\top d+\tfrac12 d^\top H^k d+cP(x^k+d) \quad\text{subject to }d_j=0\text{ for }j\notin\mathcal J^k.∇f(xk)⊤d+21​d⊤Hkd+cP(xk+d)subject to dj​=0 for j∈/Jk.

The next iterate is xk+1=xk+αkdkx^{k+1}=x^k+\alpha^k d^kxk+1=xk+αkdk. The Armijo rule tests the geometric sequence αinitkβj\alpha^k_{\mathrm{init}}\beta^jαinitk​βj and takes its largest accepted member, using the paper's decrease quantity Δk=∇f(xk)⊤dk+γ(dk)⊤Hkdk+cP(xk+dk)−cP(xk)\Delta^k=\nabla f(x^k)^\top d^k+\gamma(d^k)^\top H^kd^k+cP(x^k+d^k)-cP(x^k)Δk=∇f(xk)⊤dk+γ(dk)⊤Hkdk+cP(xk+dk)−cP(xk). Its parameters satisfy αinitk>0\alpha^k_{\mathrm{init}}>0αinitk​>0, 0<β,σ<10<\beta,\sigma<10<β,σ<1, and 0≤γ<10\le\gamma<10≤γ<1. The accepted step obeys Fc(xk+1)≤Fc(xk)+σαkΔkF_c(x^{k+1})\le F_c(x^k)+\sigma\alpha^k\Delta^kFc​(xk+1)≤Fc​(xk)+σαkΔk.

The generalized Gauss–Seidel rule means that some fixed number T≥1T\ge1T≥1 of consecutive coordinate sets covers every coordinate, starting at every iteration. The penalty is block-separable with respect to Jk\mathcal J^kJk when it is the sum of a proper closed convex function of the selected coordinates and another of their complement. A point xˉ\bar xxˉ is stationary when it lies in DDD and the one-sided directional derivative Fc′(xˉ;v)F_c'(\bar x;v)Fc′​(xˉ;v) is nonnegative for every direction v∈Rnv\in\mathbb R^nv∈Rn.

Formalization targets

Global convergence under generalized Gauss–Seidel selection

The goal is Theorem 1(e) Tseng and Yun (2009), p. 399. Under the CGD and Armijo rules, Assumption 1's uniform matrix bounds, a positive lower bound on the initial trial steps, block separability for each chosen coordinate set, and a finite upper bound on the accepted steps,

xˉ a cluster point of {xk}k≥0⟹Fc′(xˉ;v)≥0 for every v∈Rn.\bar x\text{ a cluster point of }\{x^k\}_{k\ge0} \quad\Longrightarrow\quad F_c'(\bar x;v)\ge0\text{ for every }v\in\mathbb R^n.xˉ a cluster point of {xk}k≥0​⟹Fc′​(xˉ;v)≥0 for every v∈Rn.

The mission milestones state the paper's Lemma 1, its Armijo existence claim after equation (10), Lemmas 2 and 3, and Theorem 1(a) and (b). They connect the direction subproblem, descent, stationarity and subsequence behavior to the goal. The theorem asserts stationarity of every cluster point; it does not assert that a cluster point exists or that the full sequence converges.

Significance

The result supplies a global first-order guarantee for a block update method even when the smooth part fff is nonconvex. It applies to extended-valued convex penalties, so the same statement covers both nonsmooth regularization and constraints represented by indicator functions. The bounded-window coverage condition allows varying coordinate sets rather than a fixed cyclic schedule Tseng and Yun (2009), Theorem 1(e).

The paper proves the theorem. This mission asks for a machine-checked Lean proof of that known claim and its selected supporting results. The reusable output includes a precise interface for an extended-valued penalty, the exact quadratic coordinate subproblem, Armijo backtracking with first accepted exponent, and the distinction between a convergent subsequence and a cluster point. The current draft statements compile with proof placeholders; they are proof obligations, not completed machine-checked results.

Difficulty

The descent bound alone controls decrease in objective value; it does not directly show that every coordinate has a vanishing stationarity residual. The coordinate set may vary at each iteration, and the penalty need only separate with respect to the current block. A cluster point may be reached along a subsequence that samples only one of those blocks. The argument therefore has to connect behavior along a convergent subsequence to nearby iterations within a full coverage window. The nonsmooth penalty also prevents replacing directional stationarity by a statement solely about an ordinary gradient.

Formalization scope

Vectors use EuclideanSpace ℝ (Fin n) and its Euclidean norm; Fin n starts at zero, whereas the paper numbers coordinates from one. A penalty is represented by its effective domain DDD and finite values PPP on D,usingthepublished‘ProxNewton.Inexact.IsProperClosedConvex‘definition.Everyobjectivecomparisonincludesdomainmembership;valuesoutsideD, using the published `ProxNewton.Inexact.IsProperClosedConvex` definition. Every objective comparison includes domain membership; values outside D,usingthepublished‘ProxNewton.Inexact.IsProperClosedConvex‘definition.Everyobjectivecomparisonincludesdomainmembership;valuesoutsideDareinterpretedasare interpreted asareinterpretedas+\infty.Thesmoothnessassumptionnamesanopenneighborhoodof. The smoothness assumption names an open neighborhood of .ThesmoothnessassumptionnamesanopenneighborhoodofD.Thedirection. The direction .Thedirectiond_H(x;\mathcal J)ischosenfromtheexactminimizers;alltheoremusesplaceis chosen from the exact minimizers; all theorem uses placeischosenfromtheexactminimizers;alltheoremusesplacexinininDandandandH$ positive definite. A CGD run records the chosen minimizers, so it does not rely on any unspecified value of that choice outside its valid inputs.

The Armijo predicate records the first successful geometric trial, including rejection of all earlier trials. The matrix bounds in Assumption 1 are quadratic-form bounds. Lemma 3 represents eigenvalue extremes by Rayleigh quotients on the selected coordinates; its generalized spectrum is that of the pair of principal submatrices. These extrema are used only with a nonempty block and positive-definite matrices. The paper's o(α)o(\alpha)o(α) is represented by a remainder whose quotient by α\alphaα tends to zero from the right. In Lemma 1 the line segment remains in DDD. A convergent subsequence is indexed by a strictly increasing map; a cluster point is represented separately by MapClusterPt.

No convergence of the entire iterate sequence is assumed. Stationarity requires the nonnegative directional derivative in every direction, not only the coordinates selected at one step. A valid development needs convex analysis of the subproblem, estimates for positive-definite quadratic forms, properties of lower semicontinuity and directional derivatives, and finite-dimensional subsequence arguments. Contributions proving those reusable facts and the named milestones are within scope.

Selected references

  • Paul Tseng and Sangwoon Yun, A coordinate gradient descent method for nonsmooth separable minimization, Mathematical Programming, Series B 117 (2009), 387–423. DOI: 10.1007/s10107-007-0170-0.
9 thms1 active userReviewed
Dynamic ProgrammingOptimizationProbability·Captain: mikedeng1

Negative Dynamic Programming II: With Non-Positive Rewards on Borel Spaces, the Optimal Return Satisfies the Optimality EquationResearch Paper

Motivation

Negative dynamic programming is the study of infinite-horizon sequential decision problems in which every one-stage reward is non-positive and nothing is discounted. Equivalently, a non-negative cost accumulates forever and the controller minimizes its expected total. Stochastic shortest-path problems, optimal stopping with a cost per step, and inventory or replacement models without discounting all have this form. The total cost of a policy can be infinite, and none of the contraction arguments of discounted dynamic programming apply.

Blackwell settled the discounted case on Borel state and action spaces (Blackwell 1965). There the optimal return is Borel measurable and is the unique bounded solution of the optimality equation. Strauch's paper (Strauch 1966) treats the negative case on the same Borel model. One of its main results is that the optimal return still satisfies the optimality equation, even though it need not be Borel measurable.

Timeline. Blackwell (1965): discounted case, Borel spaces. Blackwell (1967, Positive dynamic programming): positive bounded case. Strauch (1966): negative case on Borel spaces. The optimal return is absolutely measurable, (p,ε)(p,\varepsilon)(p,ε)-optimal Markov policies exist, and the optimality equation holds. Bertsekas and Shreve (1978, Stochastic Optimal Control: The Discrete-Time Case) later recast the theory with universally measurable policies, a lower semianalytic optimal cost, and Bellman's equation J∗=T(J∗)J^* = T(J^*)J∗=T(J∗) under (P), (N) and (D).

Setting

The states SSS and actions AAA are non-empty Borel sets. A law of motion q(⋅∣s,a)q(\cdot\mid s,a)q(⋅∣s,a) is a probability kernel from S×AS\times AS×A to SSS. The return r(s,a,t)r(s,a,t)r(s,a,t) is a Borel function with −∞<r≤0-\infty < r \le 0−∞<r≤0 whose expectation ∫r(s,a,t) dq(t∣s,a)\int r(s,a,t)\,dq(t\mid s,a)∫r(s,a,t)dq(t∣s,a) is finite for every (s,a)(s,a)(s,a). A policy π=(π1,π2,… )\pi = (\pi_1,\pi_2,\dots)π=(π1​,π2​,…) draws the nnnth action from a kernel πn(⋅∣s1,a1,…,sn)\pi_n(\cdot\mid s_1,a_1,\dots,s_n)πn​(⋅∣s1​,a1​,…,sn​) that may depend on the whole history. A Markov policy (f1,f2,… )(f_1,f_2,\dots)(f1​,f2​,…) uses measurable maps fn:S→Af_n:S\to Afn​:S→A, and a stationary policy f(∞)f^{(\infty)}f(∞) uses one map at every stage. The expected return from the initial state sss is

I(π)(s)=∑n≥1Esπ r(sn,an,sn+1)∈[−∞,0],I(\pi)(s) = \sum_{n\ge1} E^\pi_s\, r(s_n,a_n,s_{n+1}) \in [-\infty,0],I(π)(s)=n≥1∑​Esπ​r(sn​,an​,sn+1​)∈[−∞,0],

and the optimal return is

v∗(s)=sup⁡πI(π)(s),v^*(s) = \sup_\pi I(\pi)(s),v∗(s)=πsup​I(π)(s),

the supremum over all policies. Write M(S)M(S)M(S) for the non-positive, extended real-valued Borel functions on SSS. For u∈M(S)u\in M(S)u∈M(S) and an action aaa,

Tau(s)=∫[r(s,a,t)+u(t)] dq(t∣s,a).T_a u(s) = \int \big[r(s,a,t) + u(t)\big]\,dq(t\mid s,a).Ta​u(s)=∫[r(s,a,t)+u(t)]dq(t∣s,a).

TTT denotes the same operator for a measurable rule fff in place of aaa, and U=sup⁡nTnU = \sup_n T_nU=supn​Tn​ for a Markov policy (f1,f2,… )(f_1,f_2,\dots)(f1​,f2​,…). A policy π∗\pi^*π∗ is (p,ε)(p,\varepsilon)(p,ε)-optimal, for a probability ppp on SSS, if p{I(π∗)≥v∗−ε}=1p\{I(\pi^*) \ge v^* - \varepsilon\} = 1p{I(π∗)≥v∗−ε}=1. The Lean development uses the same names: Problem, I, In, vstar, T, Ta, U, IsPEOptimal, Conserves.

Formalization targets

Goal: the optimality equation (Theorem 8.2, negative case)

v∗(s)=sup⁡a∈ATav∗(s)for all s∈S.v^*(s) = \sup_{a\in A} T_a v^*(s) \qquad\text{for all } s\in S.v∗(s)=a∈Asup​Ta​v∗(s)for all s∈S.

The equation involves no constants and no regularity hypotheses on v∗v^*v∗.

Milestones

  1. Theorem 5.2 (a)–(g): monotonicity, translation, sup-, limit- and selection properties of UUU.
  2. Theorem 6.1: if Uv≥vUv\ge vUv≥v for v∈M(S)v\in M(S)v∈M(S), some Markov policy generated from π^\hat\piπ^ has I(π)≥v−εI(\pi)\ge v-\varepsilonI(π)≥v−ε.
  3. Lemma 6.2: UUU conserves sup⁡nI(nπ)\sup_n I({}^n\pi)supn​I(nπ) and lim⁡nUnsup⁡nI(nπ)\lim_n U^n \sup_n I({}^n\pi)limn​Unsupn​I(nπ).
  4. Theorem 6.2: one Markov policy comes within ε\varepsilonε of the supremum of the returns of countably many Markov policies.
  5. Lemma 7.1, Lemma 7.2: measurability of (s,ν)↦∫u(s,x) dν(x)(s,\nu)\mapsto\int u(s,x)\,d\nu(x)(s,ν)↦∫u(s,x)dν(x), and Borel measurability of the set of pairs (s,eπ(s))(s, e_\pi(s))(s,eπ​(s)), where eπ(s)e_\pi(s)eπ​(s) is the law of the future under π\piπ.
  6. Theorem 7.1: v∗v^*v∗ is absolutely measurable.
  7. Theorem 8.1: for every ppp and ε>0\varepsilon>0ε>0 a (p,ε)(p,\varepsilon)(p,ε)-optimal Markov policy exists.

Optional extras: Corollary 6.1 and Theorems 6.3, 6.4, 6.5 and 8.4 (bounds and characterizations for Markov policies).

Significance

The result. The optimality equation is the entry point to every structural statement about optimal policies in the negative case. With it, a stationary policy whose rule attains the supremum in sup⁡aTav∗\sup_a T_a v^*supa​Ta​v∗ can be tested for optimality. Value iteration and policy improvement can be compared with v∗v^*v∗. The gap between the negative case and the discounted and positive cases can be located precisely: in the negative case v∗v^*v∗ satisfies the equation, but it need not be its unique or extremal solution. Along the way the paper proves that v∗v^*v∗ is absolutely measurable and that nearly optimal Markov policies exist for every initial distribution. These two facts are used repeatedly in later Borel-space dynamic programming.

Formalizing it. The results are proved; none is machine-checked. A formal development would contain the first Lean treatment of a total-reward Markov decision process on Borel spaces with possibly infinite returns. It would include the policy-dependent law of the future via Ionescu-Tulcea, the measurability of the optimal return over all randomized history-dependent policies, and the Bellman equation for a non-measurable value function. Bertsekas–Shreve's version of the optimality equation (universally measurable policies) appears on the platform as a separate open statement. It concerns a different policy class and is not the statement posed here.

Difficulty

Two steps of the obvious argument fail. First, the classical proof of the optimality equation picks, at each next state ttt, a policy that is ε\varepsilonε-optimal from ttt and concatenates. Without measurable selection this concatenation is not a policy. The set of ttt where a given policy is ε\varepsilonε-optimal need not be Borel, and v∗v^*v∗ itself need not be Borel measurable, so even ∫v∗ dq\int v^*\,dq∫v∗dq needs justification. Second, the contraction argument of the discounted case is unavailable. UUU does not contract, it conserves v+cv+cv+c along with vvv, and value iteration from 000 can converge to a function strictly above v∗v^*v∗. The paper's Example 6.1 shows that the limit of Un0U^n0Un0, the best return of generated Markov policies, and the best stationary return can all differ. Any argument has to work with policies that are only nearly optimal, and only outside sets of measure zero that depend on the initial distribution.

Formalization scope

  • States and actions are non-empty standard Borel types. Policies are the published Blackwell plans (DiscountedDP.Stationary.Plan): randomized and history-dependent, one Markov kernel per decision, with decisions numbered from 000 (Lean's π.κ n is the paper's πn+1\pi_{n+1}πn+1​). Markov policies are MarkovPlan; "π\piπ-generated" and G(π^)G(\hat\pi)G(π^) are the published IsGenerated and IsGeneratedPlan.
  • Only the negative case is formalized: r≤0r\le0r≤0 real-valued, qrqrqr integrable, β=1\beta=1β=1. The discounted and positive parts of Theorems 5.2, 7.1, 8.1 and 8.2, including the uniqueness and minimality clauses of Theorem 8.2, are not stated.
  • Returns take values in EReal and are computed as minus a lower Lebesgue integral (lintegral) of the non-negative loss. A Bochner integral is never used for a return: it would map −∞-\infty−∞ to 000 and make the goal false or vacuous.
  • v∗v^*v∗ is the supremum over all plans. It is not assumed measurable and is never replaced by a measurable modification. Tav∗T_a v^*Ta​v∗ is a lintegral of a possibly non-measurable function, which equals the completion integral because v∗v^*v∗ is absolutely measurable (Theorem 7.1).
  • Explicit readings of the page: the hypothesis u∈M(S)u\in M(S)u∈M(S) of each operator statement is IsNegM u. Theorem 5.2(b) assumes u+c∈M(S)u+c\in M(S)u+c∈M(S). "UUU conserves vvv" includes v∈M(S)v\in M(S)v∈M(S). Lemma 6.2 asserts that the limit defining vπv_\pivπ​ exists. Theorem 6.2 states ≥\ge≥ everywhere and >>> where sup⁡jI(πj)>−∞\sup_j I(\pi^j) > -\inftysupj​I(πj)>−∞ (the printed >>> fails where the supremum is −∞-\infty−∞). (p,ε)(p,\varepsilon)(p,ε)-optimality is "ppp-almost everywhere". Absolute measurability is NullMeasurable for every probability measure. P(X)P(X)P(X) carries the Giry σ-field. The extra Theorem 8.4 reads the printed "I(π)<uI(\pi) < uI(π)<u" as "≤\le≤" (the strict reading is false).
  • Reusable infrastructure: the negative model, the Ionescu-Tulcea law of the future (futureLaw), and the measurability lemmas for integrals against random measures. Contributions of measurable-selection and analytic-set results (projections of Borel sets are universally measurable; von Neumann selection) are welcome and needed.

Selected references

  • R. E. Strauch, Negative Dynamic Programming, Ann. Math. Statist. 37(4) (1966) 871–890. https://doi.org/10.1214/aoms/1177699369
  • D. Blackwell, Discounted Dynamic Programming, Ann. Math. Statist. 36(1) (1965) 226–235. https://doi.org/10.1214/aoms/1177700285
  • D. Blackwell, Positive Dynamic Programming, Proc. Fifth Berkeley Symp. Math. Statist. Probab. 1 (1967) 415–418. https://projecteuclid.org/euclid.bsmsp/1200512999
  • L. E. Dubins and L. J. Savage, How to Gamble If You Must, McGraw-Hill, 1965.
  • D. P. Bertsekas and S. E. Shreve, Stochastic Optimal Control: The Discrete-Time Case, Academic Press, 1978. https://web.mit.edu/dimitrib/www/soc.html
21 thms1 active userReviewed
OptimizationProbability·Captain: mikedeng1

Bounded Rationality in Newsvendor Models I: Under Uniform Demand the Logit Newsvendor's Order Is Truncated Normal and Its Mean Is Pulled from x* Toward the MidpointResearch Paper

Motivation

The newsvendor problem is the basic single-period inventory model: a decision maker orders xxx units at unit cost ccc before a random demand DDD is observed, sells min⁡(D,x)\min(D, x)min(D,x) units at price ppp, and loses unsold units. Its optimal order is the critical fractile x∗=F−1(1−c/p)x^* = F^{-1}(1 - c/p)x∗=F−1(1−c/p) of the demand distribution FFF. Laboratory experiments show that human subjects do not order x∗x^*x∗. Schweitzer and Cachon (Management Science 46(3), 2000) gave subjects uniform demand on [0,300][0, 300][0,300] and found that the average order lay below x∗x^*x∗ for a high-margin product and above x∗x^*x∗ for a low-margin product: orders are pulled toward the center of the demand range.

Su (Manufacturing & Service Operations Management 10(4), 2008) explains this pattern with a model of bounded rationality: the decision maker does not pick the best order with certainty, but picks better orders more often, through a logit choice rule. This mission formalizes the first results of that paper, the case of uniform demand, where the logit order has an explicit law and the pull toward the center can be proved exactly. The source is the published MSOM 2008 article; all page numbers are its printed pages.

Setting

A decision maker with utility uuu over an interval S⊆RS \subseteq \mathbb RS⊆R and bounded-rationality parameter β>0\beta > 0β>0 makes a random choice YYY with the logit density

ψ(y)=eu(y)/β∫Seu(v)/β dv,y∈S,\psi(y) = \frac{e^{u(y)/\beta}}{\int_S e^{u(v)/\beta}\,dv}, \qquad y \in S,ψ(y)=∫S​eu(v)/βdveu(y)/β​,y∈S,

and ψ(y)=0\psi(y) = 0ψ(y)=0 off SSS (eq. (2), p. 571). Large β\betaβ means noisy choices; small β\betaβ concentrates the choice near the maximizer of uuu.

In the newsvendor problem the price ppp and unit cost ccc satisfy 0<c<p0 < c < p0<c<p. Demand DDD has density fff and distribution function FFF. The expected profit of ordering xxx units is

π(x)=p Emin⁡(D,x)−cx(eq. (3), p. 572),\pi(x) = p\,\mathbb E\min(D, x) - c x \qquad \text{(eq. (3), p. 572)},π(x)=pEmin(D,x)−cx(eq. (3), p. 572),

and the optimal solution x∗x^*x∗ is its maximizer. The behavioral solution X♭X^\flatX♭ is the logit choice with utility u=πu = \piu=π over the decision domain SSS, the smallest interval containing the support of fff (eq. (4), p. 572).

This mission takes uniform demand D∼U[a,b]D \sim U[a, b]D∼U[a,b] with b>a≥0b > a \ge 0b>a≥0: f=1/(b−a)f = 1/(b-a)f=1/(b−a) on [a,b][a, b][a,b] and S=[a,b]S = [a, b]S=[a,b]. The midpoint of the demand range is m=(a+b)/2m = (a + b)/2m=(a+b)/2. Two parameters recur:

μ=b−cp(b−a),σ2=β b−ap.\mu = b - \frac{c}{p}(b - a), \qquad \sigma^2 = \beta\,\frac{b-a}{p}.μ=b−pc​(b−a),σ2=βpb−a​.

A truncated normal law on [a,b][a, b][a,b] with parameters μ,σ2\mu, \sigma^2μ,σ2 has density proportional to e−(x−μ)2/2σ2e^{-(x-\mu)^2/2\sigma^2}e−(x−μ)2/2σ2 on [a,b][a, b][a,b] and zero elsewhere (eq. (37), p. 586). Write ϕ\phiϕ and Φ\PhiΦ for the standard normal density and distribution function.

Formalization targets

Goal: Proposition 3 (p. 577), midpoint bias

x∗>m  ⟹  EX♭<x∗,x∗<m  ⟹  EX♭>x∗.x^* > m \implies \mathbb E X^\flat < x^*, \qquad x^* < m \implies \mathbb E X^\flat > x^*.x∗>m⟹EX♭<x∗,x∗<m⟹EX♭>x∗.

The goal is stated for every β>0\beta > 0β>0 and every 0<c<p0 < c < p0<c<p, with x∗x^*x∗ any maximizer of π\piπ over R\mathbb RR; it fixes only the sign of the bias, not its size.

Milestones

  1. Eq. (5), p. 572. On [a,b][a, b][a,b], π(x)=Ax2+Bx+C\pi(x) = Ax^2 + Bx + Cπ(x)=Ax2+Bx+C with A=−p/(2(b−a))A = -p/(2(b-a))A=−p/(2(b−a)), B=pb/(b−a)−cB = pb/(b-a) - cB=pb/(b−a)−c, C=−pa2/(2(b−a))C = -pa^2/(2(b-a))C=−pa2/(2(b−a)).
  2. The optimum, pp. 572–573. π\piπ is maximized over R\mathbb RR exactly at μ\muμ, and F(μ)=1−c/pF(\mu) = 1 - c/pF(μ)=1−c/p, so x∗=F−1(1−c/p)=μx^* = F^{-1}(1 - c/p) = \mux∗=F−1(1−c/p)=μ.
  3. Proposition 1, pp. 572–573. The density of X♭X^\flatX♭ equals the truncated normal density on [a,b][a,b][a,b] with parameters μ\muμ and σ2\sigma^2σ2.
  4. Corollary 1, p. 573.
EX♭=μ−σ ϕ((b−μ)/σ)−ϕ((a−μ)/σ)Φ((b−μ)/σ)−Φ((a−μ)/σ).(8)\mathbb E X^\flat = \mu - \sigma\,\frac{\phi((b-\mu)/\sigma) - \phi((a-\mu)/\sigma)}{\Phi((b-\mu)/\sigma) - \Phi((a-\mu)/\sigma)}. \qquad (8)EX♭=μ−σΦ((b−μ)/σ)−Φ((a−μ)/σ)ϕ((b−μ)/σ)−ϕ((a−μ)/σ)​.(8)

Significance

Proposition 3 gives a single mechanism, noise in the choice of the order, that produces both directions of the bias observed by Schweitzer and Cachon: under uniform demand, a high-profit product (x∗>mx^* > mx∗>m) is underordered on average and a low-profit product (x∗<mx^* < mx∗<m) is overordered. Proposition 1 gives the full law of the boundedly rational order, which is what makes the model testable: the paper fits the truncated normal law to experimental order data and estimates β\betaβ (§5). Corollary 1 is the closed-form mean used in that fit and in Proposition 3.

These results are proved in the paper by direct computation; none of them has a machine-checked proof. The mission adds a checked development of the continuous logit choice model on an interval, of the truncated normal law and its mean, and of the uniform-demand newsvendor profit, together with exact statements of what the paper's informal phrases mean (see Formalization scope).

Difficulty

Each step is elementary on paper, but the formal content sits in the integrals. Expected sales Emin⁡(D,x)\mathbb E\min(D, x)Emin(D,x) must be computed as a piecewise integral to obtain eq. (5), and the maximizer must be identified over all of R\mathbb RR, not only on [a,b][a, b][a,b], which needs the profit outside the support as well. Proposition 1 is an identity between two normalized densities, so both normalizing integrals must be shown positive and finite. Corollary 1 requires the mean of a truncated Gaussian in terms of Mathlib's Gaussian density and distribution function, which is not in Mathlib. For Proposition 3, the natural shortcut "the mean of a truncated normal is μ\muμ" is false whenever μ≠m\mu \ne mμ=m, and it is exactly the asymmetry of the truncation that produces the bias.

Formalization scope

All quantities are real numbers, with the paper's standing assumptions as hypotheses: b>a≥0b > a \ge 0b>a≥0, 0<c<p0 < c < p0<c<p (the page states p>cp > cp>c; c>0c > 0c>0 is read so that x∗x^*x∗ lies inside (a,b)(a, b)(a,b)), and β>0\beta > 0β>0 (eq. (2) divides by β\betaβ; β=0\beta = 0β=0, perfect rationality, is a limit, not a value of the formula). The decision domain is the closed interval [a,b][a, b][a,b]; the logit density is zero off it, and expectations are integrals over R\mathbb RR. The parameter σ\sigmaσ is the positive square root of β(b−a)/p\beta(b-a)/pβ(b−a)/p, and ϕ\phiϕ, Φ\PhiΦ are Mathlib's gaussianPDFReal 0 1 and the distribution function of gaussianReal 0 1.

Corrections and readings relative to the printed text:

  • Eq. (5) is stated for x∈[a,b]x \in [a, b]x∈[a,b] only; outside it π\piπ is linear.
  • Proposition 1 says the truncated normal has "mean μ\muμ and variance σ2\sigma^2σ2"; these are the parameters before truncation (as in eq. (37)), and the statement is the equality of densities. The mean of X♭X^\flatX♭ is (8), not μ\muμ.
  • In Proposition 3, "fff constant over [a,b][a, b][a,b]" is read as uniform demand on [a,b][a,b][a,b], as in its proof. The conclusions are inequalities on EX♭\mathbb E X^\flatEX♭, because the §6 preamble (p. 577) defines "overorders" and "underorders" with the inequalities swapped relative to the proof (p. 586); the formal statement follows the proof.

The optimal solution x∗x^*x∗ in the goal is a hypothesis that x∗x^*x∗ maximizes π\piπ over R\mathbb RR, never a variable set to a closed form by fiat; milestone 2 shows the hypothesis is satisfiable and identifies x∗x^*x∗. No statement holds vacuously through a junk value: on [a,b][a, b][a,b] with a<ba < ba<b the logit normalizer is the integral of a continuous positive function over an interval of positive length, and the denominator of (8) is positive because σ>0\sigma > 0σ>0.

Reusable beyond this mission: the logit density on an interval, the truncated normal density, and its mean formula. Proofs of any milestone, and of the mean of a truncated normal law in general, are welcome.

Selected references

  • X. Su, Bounded Rationality in Newsvendor Models, Manufacturing & Service Operations Management 10(4):566–589, 2008. https://doi.org/10.1287/msom.1070.0200
  • M. E. Schweitzer, G. P. Cachon, Decision Bias in the Newsvendor Problem with a Known Demand Distribution: Experimental Evidence, Management Science 46(3):404–420, 2000. https://doi.org/10.1287/mnsc.46.3.404.12070
8 thms1 active userReviewed
ProbabilityStochastic Systems·Captain: mikedeng1

Understanding the Efficiency of Multi-Server Service Systems II: In the M/M/s Queue with (1 − ρ)√s = γ, the Wait of a Delayed Customer Is Exponential with Mean 1/(γ√s)Research Paper

Motivation

A service system with several servers can use a larger fraction of its capacity than a single-server system while still giving customers an acceptable wait. The design question is how much unused capacity is needed as the number of servers grows. Ward Whitt's study begins with a factory manager deciding whether to put four machines in one work area. It uses queueing models to connect the number of machines, their utilization, and the delays customers experience. The proposed utilization rule, (1−ρ)s=γ(1-\rho)\sqrt{s}=\gamma(1−ρ)s​=γ, keeps the unused fraction 1−ρ1-\rho1−ρ proportional to 1/s1/\sqrt{s}1/s​ as the number of servers sss changes (Whitt 1992, pp. 708–710).

The paper distinguishes the chance that a customer waits at all from the length of the wait once a delay occurs. That distinction matters when a staffing choice must control a long wait rather than merely the fraction of customers who wait. The exact M/M/sM/M/sM/M/s result in §3.1 gives a benchmark for the paper's later approximations to more general arrival and service processes (Whitt 1992, pp. 719–720).

Setting

An M/M/sM/M/sM/M/s queue has Poisson arrivals at rate λ\lambdaλ, sss identical servers, independent exponential service times, unlimited waiting room, and first-come first-served service. Time is measured so that each server's service rate is one. The server utilization is ρ=λ/s\rho=\lambda/sρ=λ/s. We consider s≥1s\geq1s≥1 and 0<λ<s0<\lambda<s0<λ<s, so the queue has a stationary distribution. Let pnp_npn​ be the stationary probability that nnn customers are in the system. Its state process is a birth–death process: arrivals occur at rate λ\lambdaλ and departures from state nnn occur at rate min⁡(n,s)\min(n,s)min(n,s).

Let WWW be an arriving customer's steady-state time in the queue before service begins. An arrival finding fewer than sss customers starts service at once. An arrival finding s+js+js+j customers waits for j+1j+1j+1 service completions while all servers are occupied. In the formal model, the law of WWW has an atom at zero of mass ∑n<spn\sum_{n<s}p_n∑n<s​pn​ and, for each j≥0j\geq0j≥0, a gamma component with shape j+1j+1j+1, rate sss, and weight ps+jp_{s+j}ps+j​. The arrival-state weights use the Poisson-arrivals-see-time-averages property. This construction represents the FCFS waiting time independently of any claim about its eventual conditional distribution.

For an event W>0W>0W>0 of positive probability, L(W∣W>0)\mathcal L(W\mid W>0)L(W∣W>0) denotes the conditional waiting-time law. The paper calls a customer delayed exactly when W>0W>0W>0. The scale parameter γ\gammaγ is defined by the utilization equation (1−ρ)s=γ(1-\rho)\sqrt{s}=\gamma(1−ρ)s​=γ; because ρ<1\rho<1ρ<1, this equation makes γ\gammaγ positive.

Formalization targets

The supporting target is the §3.1 statement that a delayed customer's waiting time is exponential with rate s(1−ρ)s(1-\rho)s(1−ρ), equivalently with mean 1/[s(1−ρ)]1/[s(1-\rho)]1/[s(1−ρ)]:

L(W∣W>0)=Exp⁡(s(1−ρ)).\mathcal L(W\mid W>0)=\operatorname{Exp}\bigl(s(1-\rho)\bigr).L(W∣W>0)=Exp(s(1−ρ)).

The goal is Proposition 3.1 under the utilization equation. It records both the entire conditional tail and its mean, for every x≥0x\geq0x≥0:

P(W>x∣W>0)=e−γs x,E[W∣W>0]=1γs.\mathbb P(W>x\mid W>0)=e^{-\gamma\sqrt{s}\,x}, \qquad \mathbb E[W\mid W>0]=\frac{1}{\gamma\sqrt{s}}.P(W>x∣W>0)=e−γs​x,E[W∣W>0]=γs​1​.

The goal also asserts P(W>0)>0\mathbb P(W>0)>0P(W>0)>0, so these conditional quantities are defined. The scan of Proposition 3.1 prints the exponent without xxx and the mean denominator with 2\sqrt22​; the statement here uses the values determined by the preceding exponential-law sentence and by the paragraph following the proposition (Whitt 1992, pp. 719–720).

Significance

The result gives an exact delay distribution for the Markovian multi-server system. With sss servers and the utilization equation held fixed, the conditional tail at any positive threshold and the mean wait of a delayed customer both depend on sss through γs\gamma\sqrt{s}γs​. Thus the same rule that organizes the probability-of-delay discussion has a definite implication for customers who actually wait. It is also the exact comparison point for the paper's later, explicitly approximate analysis of general G/G/sG/G/sG/G/s queues (Whitt 1992, §3.2).

Formalizing this result supplies a reusable representation of the stationary FCFS waiting-time law as a mixture of Erlang distributions, together with a precise connection between birth–death state probabilities and delay. The related platform items QueueingFundamentals.BirthDeath.mmc_steady_state and QueueingFundamentals.BirthDeath.erlang_c_formula state the stationary state probabilities and the probability of delay, respectively; both are open and neither states this waiting-time law. The open QueueingFundamentals.BirthDeath.halfin_whitt concerns the heavy-traffic limit for the M/M/cM/M/cM/M/c case, a different target. The current result is known mathematically from Whitt's 1992 paper; this mission asks for its machine-checked proof in the specified model.

Difficulty

The conditional distribution is not immediate from an individual service time. A delayed arrival can encounter any queue length s+js+js+j, so its waiting time is a gamma law whose shape varies with jjj. The stationary probabilities provide the weights of infinitely many such components. Showing that this whole mixture has one exponential law requires a relation among the stationary tail probabilities and control of the infinite sum. The same mixture must have a well-defined first moment for the conditional mean formula.

Formalization scope

Lean uses a natural number s≥1s\geq1s≥1, a real arrival rate 0<λ<s0<\lambda<s0<λ<s, and a real sequence pnp_npn​ satisfying the published QueueingFundamentals.BirthDeath.IsSteadyState predicate with constant birth rate λ\lambdaλ and death rate min⁡(n,s)\min(n,s)min(n,s). That predicate includes nonnegativity, total mass one, and the global balance equations. Its published definition is imported as a reference. Service rate is exactly one, as fixed in §2.1 of the paper. The conditional waiting-time law is a measure on real time, with a point mass at zero and a countable gamma mixture. The positive-time restriction of this measure expresses conditioning in the milestone; real measure values and an integral express the tail and mean in the goal.

The formal hypotheses spell out positive arrival rate, at least one server, and subcritical utilization. These make the stationary law and the positive exponential rate meaningful. The goal states positive delay probability before dividing by it. The waiting-time law is built from stationary queue lengths and service completions, rather than being defined as exponential; proving that it becomes exponential is the mathematical task. Solvers may develop reusable measure-mixture identities, stationary-tail facts, and gamma or exponential distribution results. The approximation formulas of §3.2 and the paper's numerical tables are outside this mission.

Selected references

  • Ward Whitt, Understanding the Efficiency of Multi-Server Service Systems, Management Science 38(5):708–723, 1992. DOI: 10.1287/mnsc.38.5.708.
4 thms1 active userReviewed
CombinatoricsGraph TheoryLinear Optimization+1·Captain: mikedeng1

Approximating Minimum Bounded Degree Spanning Trees to within One of Optimal 1: Iterative Rounding Finds a Spanning Tree of Cost at Most the LP Optimum with Every Degree at Most B_v + 1Research Paper

Motivation

A network designer may need a cheap connection that keeps the number of links at each site within a local capacity. The minimum bounded-degree spanning tree problem asks for the cheapest spanning tree of a weighted, undirected graph whose vertex degrees respect prescribed upper bounds. The capacity constraints matter in settings where a low-cost tree alone can overload a few hubs. They also make the optimization problem difficult: even with unit edge weights and a bound of two at every vertex, feasibility includes the Hamiltonian path problem. Singh and Lau study the useful relaxation in which the returned tree may exceed each degree bound by one while its cost stays no greater than the best tree satisfying every original bound Singh–Lau 2007.

The weighted result settled the bounded-degree form of a conjecture discussed in their paper. Goemans had obtained a cost-preserving degree guarantee of Bv+2B_v+2Bv​+2; an earlier unweighted local-search result attained a +1+1+1 degree allowance but did not cover arbitrary real costs. Singh and Lau's Theorem 1.2 gives the +1+1+1 allowance in the weighted setting, without assuming nonnegative costs or triangle inequalities Singh–Lau 2007, pp. 661–662. The platform's open WilliamsonShmoys.minimum_degree_tree_local_search_approximation concerns the related unweighted minimum-degree local-search problem, with a different objective and guarantee.

Setting

Let VVV be a finite vertex set and EEE a finite set of unordered, non-loop edges. A spanning tree T⊆ET\subseteq ET⊆E connects every vertex without a cycle. Every edge eee has a real cost cec_ece​, so c(T)=∑e∈Tcec(T)=\sum_{e\in T}c_ec(T)=∑e∈T​ce​. Each vertex vvv has an integer upper bound BvB_vBv​ on its tree degree dT(v)d_T(v)dT​(v). A feasible original solution has dT(v)≤Bvd_T(v)\le B_vdT​(v)≤Bv​ for all vvv.

The paper studies a broader connecting-tree problem. A forest FFF is already selected; its connected components, including isolated vertices, are supernodes. The remaining graph has edges EEE disjoint from FFF. A set H⊆EH\subseteq EH⊆E is an FFF-tree if H∪FH\cup FH∪F is a spanning tree. Degree bounds are active only at vertices in W⊆VW\subseteq VW⊆V, and dH(v)d_H(v)dH​(v) counts edges of HHH, not edges of FFF. The initial spanning-tree problem is the case F=∅F=\varnothingF=∅ and W=VW=VW=V Singh–Lau 2007, p. 665.

For an edge set QQQ and vertex set SSS, write Q(S)Q(S)Q(S) for edges of QQQ with both endpoints in SSS, and δE(v)\delta_E(v)δE​(v) for edges of EEE incident to vvv. The LP-MBDCT relaxation has a nonnegative variable xex_exe​ for each e∈Ee\in Ee∈E. It requires x(E)=∣V∣−∣F∣−1x(E)=|V|-|F|-1x(E)=∣V∣−∣F∣−1, requires x(E(S))≤∣S∣−∣F(S)∣−1x(E(S))\le |S|-|F(S)|-1x(E(S))≤∣S∣−∣F(S)∣−1 for every nonempty union SSS of supernodes, and requires x(δE(v))≤Bvx(\delta_E(v))\le B_vx(δE​(v))≤Bv​ for v∈Wv\in Wv∈W. Its objective is ∑e∈Ecexe\sum_{e\in E}c_ex_e∑e∈E​ce​xe​. At F=∅F=\varnothingF=∅, W=VW=VW=V, this is the spanning-tree LP in equations (1)–(5) Singh–Lau 2007, pp. 662, 665.

Formalization targets

Theorem 1.2: bounded-degree spanning trees

For every feasible initial LP, the paper's Figure 4 algorithm has a terminating run. Every possible run returns a spanning tree TTT satisfying

dT(v)≤Bv+1(v∈V),c(T)≤∑e∈Ecexefor every LP-feasible x.d_T(v)\le B_v+1\quad(v\in V),\qquad c(T)\le\sum_{e\in E}c_ex_e\quad\text{for every LP-feasible }x.dT​(v)≤Bv​+1(v∈V),c(T)≤e∈E∑​ce​xe​for every LP-feasible x.

In particular, c(T)c(T)c(T) is at most the cost of every spanning tree satisfying the original bounds, and thus at most the original optimum. Every run has at most 2∣V∣−12|V|-12∣V∣−1 iterations. The cost comparison against every LP-feasible vector states the stronger bound used by Theorem 4.2 Singh–Lau 2007, pp. 661, 666.

Theorem 4.2 and its milestones

The connecting-tree theorem makes the same cost and degree guarantee for an arbitrary feasible state (E,B,W,F)(E,B,W,F)(E,B,W,F), with dH(v)≤Bv+1d_H(v)\le B_v+1dH​(v)≤Bv​+1 only for v∈Wv\in Wv∈W. The attack path follows the paper's numbered results: Lemma 4.3 describes tight constraints by a laminar family, Claim 4.5 classifies vertices with one excess token, Lemma 4.6 bounds tokens inside a laminar subtree, and Lemma 4.1 says each nonterminal basic solution has either a unit-valued edge or a removable degree constraint. Theorem 4.2 then supplies the general guarantee Singh–Lau 2007, pp. 666–667.

Significance

The result allows one unit of violation at each local degree bound while preserving the cost benchmark exactly. It applies even when negative edge costs make standard multiplicative cost ratios awkward. The connecting-tree statement is useful beyond the initial instance because it covers a forest already accumulated and a subset of degree constraints still active.

This mission formalizes the known 2007 result as an open Lean proof target; the paper proves the mathematics, while the declarations here currently have no machine-checked proofs. A completed development would provide a reusable account of a spanning-tree LP with forest contraction, basic solutions defined by tight rows, and a finite relational algorithm with all choices exposed. These objects could support the paper's lower-and-upper-bound extension and other iterative-rounding results.

Difficulty

Rounding an arbitrary fractional edge upward can increase cost beyond the LP benchmark. Restricting the algorithm to edges with value exactly one protects that benchmark, but a nonterminal basic solution need not present a unit-valued edge. The central structural question is then whether some active degree constraint can be removed with only a one-unit effect on the final degree. The tight subtour rows and degree rows overlap, so counting edges from the support alone is insufficient; Lemmas 4.3–4.6 resolve the dependency and token-counting issues that make the direct argument fail Singh–Lau 2007, pp. 666–667.

Formalization scope

Lean represents edges as Sym2 V in finite sets, excluding diagonal edges. It represents LP vectors on all unordered pairs but requires zero values outside the current edge set. A forest is acyclic on the full vertex set. A compatible subtour set is a union of its connected components. Subtour right-hand sides are real numbers, so subtraction does not truncate at zero; the empty set is excluded because the literal row would make the LP infeasible. Degree bounds are integers so the algorithm's decrements are total. A basic feasible solution is determined uniquely by its tight equality rows among vectors supported on the current edge set. Linear independence in Lemma 4.3 is checked on the positive support.

Figure 4 is a relation because the LP optimum, a unit-valued edge, and a qualifying vertex may each be chosen in several ways. Its edge and vertex tests are mandatory when their conditions hold. The vertex test uses the support before deleting the selected edge and bounds after decrementing its endpoints, following the printed order. “The algorithm returns” is represented by existence of a run, an iteration bound on every step chain, and guarantees for every returned set. “Cost at most the LP optimum” means cost no greater than the objective at every feasible LP vector. “Polynomial time” is represented by the iteration count; the ellipsoid LP solver and separation-oracle complexity are outside this mission. Lemma 4.6's token distribution is expressed as its explicit counting inequality.

The goal asserts the Figure 4 output and its LP cost guarantee. Bare existence of a tree with cost at most the optimum and degrees at most Bv+1B_v+1Bv​+1 would be witnessed by an original optimal tree and would lose the algorithmic theorem. A return guarantee without run existence would hold vacuously for a stuck relation. Contributions that prove the stated structural lemmas, establish the LP oracle's basic-optimum availability, or close the algorithm invariant are within scope. No existing platform item treats this bounded-degree LP and iterative-rounding algorithm under these conventions; the plain minimum-spanning-tree LP definition uses a different edge representation and no forest or degree constraints.

Selected references

  • Mohit Singh and Lap Chi Lau, Approximating Minimum Bounded Degree Spanning Trees to within One of Optimal, Proceedings of the 39th ACM Symposium on Theory of Computing (STOC), 2007, pp. 661–670. DOI: 10.1145/1250790.1250887.
8 thms1 active userReviewed
ProbabilityStochastic Systems·Captain: mikedeng1

The G/GI/N Queue in the Halfin–Whitt Regime 3: The Number in a Non-Idling G/GI/N System Satisfies the Infinite-Server System Equation (2.8)Research Paper

Motivation

A many-server queue has NNN identical servers, a stream of arriving customers, and a buffer in which customers wait when every server is busy. It models call centres, hospital wards and server farms, and the regime of practical interest is the Halfin–Whitt or quality-and-efficiency-driven regime, in which the number of servers NNN grows with the arrival rate λN\lambda_NλN​ so that N(1−λN/N)→β\sqrt N(1-\lambda_N/N)\to\betaN​(1−λN​/N)→β: servers are almost fully used, yet only a vanishing fraction of customers waits.

  • Halfin and Whitt (1981, Oper. Res. 29) identified the regime and proved a diffusion limit for exponential service times (GI/M/NGI/M/NGI/M/N).
  • Puhalskii and Reiman (2000, Adv. Appl. Probab. 32) treated phase-type service distributions (GI/PH/NGI/PH/NGI/PH/N).
  • Reed (2009, arXiv:0912.2837), the source of this mission, obtained the fluid and diffusion limits of the number in system for the G/GI/NG/GI/NG/GI/N queue with a general service-time distribution, assuming only a finite mean. Its approach rests on a sample-path identity, the system equation (2.8), which writes the number in system of the NNN-server queue through quantities of an infinite-server queue fed by the same arrivals.

This mission formalizes that identity and the proposition that yields it.

Setting

Fix one realisation of the queue (Reed, §2, p. 6).

  • There are NNN servers. At time 0−0-0− there are Q0Q_0Q0​ customers; the first min⁡(Q0,N)\min(Q_0,N)min(Q0​,N) are in service, with residual service times η~i\tilde\eta_iη~​i​, and the other (Q0−N)+(Q_0-N)^+(Q0​−N)+ wait.
  • Customers then arrive at times 0≤τ1≤τ2≤⋯0\le\tau_1\le\tau_2\le\cdots0≤τ1​≤τ2​≤⋯ (with τ0=0\tau_0=0τ0​=0); the arrival process is A(t)=#{i≥1:τi≤t}A(t)=\#\{i\ge1:\tau_i\le t\}A(t)=#{i≥1:τi​≤t}. Arrivals at time 000 are counted in A(0)A(0)A(0), so in general Q(0)≠Q0Q(0)\ne Q_0Q(0)=Q0​.
  • The iii-th customer to enter service after time 0−0-0− has service time ηi\eta_iηi​. Service is first come first served, so the first (Q0−N)+(Q_0-N)^+(Q0​−N)+ of these are the initial waiting customers and the iii-th arrival has service time η(Q0−N)++i\eta_{(Q_0-N)^++i}η(Q0​−N)++i​.
  • wi≥0w_i\ge0wi​≥0 is the waiting time of the iii-th arrival and w~i≥0\tilde w_i\ge0w~i​≥0 that of the initial customer N+iN+iN+i.
  • FFF is the service-time distribution, with tail G=1−FG=1-FG=1−F, and F0F_0F0​ the residual service-time distribution, with tail Fˉ0\bar F_0Fˉ0​; both are carried by [0,∞)[0,\infty)[0,∞).

The number in system is (2.2):

Q(t)=∑i=1min⁡(Q0,N)1{η~i>t}+∑i=1(Q0−N)+1{w~i+ηi>t}+∑i=1A(t)1{τi+wi+η(Q0−N)++i>t}.Q(t)=\sum_{i=1}^{\min(Q_0,N)}1\{\tilde\eta_i>t\}+\sum_{i=1}^{(Q_0-N)^+}1\{\tilde w_i+\eta_i>t\}+\sum_{i=1}^{A(t)}1\{\tau_i+w_i+\eta_{(Q_0-N)^++i}>t\}.Q(t)=i=1∑min(Q0​,N)​1{η~​i​>t}+i=1∑(Q0​−N)+​1{w~i​+ηi​>t}+i=1∑A(t)​1{τi​+wi​+η(Q0​−N)++i​>t}.

Centring each indicator at its (conditional) mean produces the terms

W0(t)=∑i=1min⁡(Q0,N)(1{η~i>t}−Fˉ0(t)),AG(t)=∫0tG(t−s) dA(s),I(t)=min⁡(Q0,N)Fˉ0(t)+(Q0−N)+G(t),W_0(t)=\sum_{i=1}^{\min(Q_0,N)}\big(1\{\tilde\eta_i>t\}-\bar F_0(t)\big),\qquad A_G(t)=\int_0^tG(t-s)\,dA(s),\qquad I(t)=\min(Q_0,N)\bar F_0(t)+(Q_0-N)^+G(t),W0​(t)=i=1∑min(Q0​,N)​(1{η~​i​>t}−Fˉ0​(t)),AG​(t)=∫0t​G(t−s)dA(s),I(t)=min(Q0​,N)Fˉ0​(t)+(Q0​−N)+G(t),

and M2(t)M_2(t)M2​(t), the analogous centred sum (2.5) over the customers who entered service after time 0−0-0−. AG(t)A_G(t)AG​(t) is the conditional mean number in a G/GI/∞G/GI/\inftyG/GI/∞ queue fed by the same arrivals.

A sample path is non-idling if for every t≥0t\ge0t≥0

(Q(t)−N)+=∑i=1(Q0−N)+1{t<w~i}+∑i=1A(t)1{τi≤t<τi+wi},(Q(t)-N)^+=\sum_{i=1}^{(Q_0-N)^+}1\{t<\tilde w_i\}+\sum_{i=1}^{A(t)}1\{\tau_i\le t<\tau_i+w_i\},(Q(t)−N)+=i=1∑(Q0​−N)+​1{t<w~i​}+i=1∑A(t)​1{τi​≤t<τi​+wi​},

that is, the customers waiting at time ttt are exactly those who have arrived and not yet entered service. This holds for every first-come-first-served NNN-server path that never idles a server while a customer waits; the paper uses it at the start of the proof of Proposition 2.1 (p. 8).

Formalization targets

Goal: the system equation (2.8), p. 9

For a non-idling sample path and every t≥0t\ge 0t≥0,

Q(t)=I(t)+W0(t)+M2(t)+AG(t)+∫0t(Q(t−s)−N)+ dF(s).Q(t)=I(t)+W_0(t)+M_2(t)+A_G(t)+\int_0^t\big(Q(t-s)-N\big)^+\,dF(s).Q(t)=I(t)+W0​(t)+M2​(t)+AG​(t)+∫0t​(Q(t−s)−N)+dF(s).

Milestones

  1. The first two equalities of the proof of Proposition 2.1 (p. 8): ∑i≤A(t)(G(t−τi−wi)−G(t−τi))=∑i≤A(t)∫0∞1{t−(τi+wi)<s≤t−τi} dF(s)\sum_{i\le A(t)}\big(G(t-\tau_i-w_i)-G(t-\tau_i)\big)=\sum_{i\le A(t)}\int_0^\infty 1\{t-(\tau_i+w_i)<s\le t-\tau_i\}\,dF(s)∑i≤A(t)​(G(t−τi​−wi​)−G(t−τi​))=∑i≤A(t)​∫0∞​1{t−(τi​+wi​)<s≤t−τi​}dF(s).
  2. The "reverse argument" of the proof (p. 9): ∫0t∑i≤(Q0−N)+1{w~i>t−s} dF(s)=∑i≤(Q0−N)+(G(t−w~i)−G(t))\int_0^t\sum_{i\le(Q_0-N)^+}1\{\tilde w_i>t-s\}\,dF(s)=\sum_{i\le(Q_0-N)^+}\big(G(t-\tilde w_i)-G(t)\big)∫0t​∑i≤(Q0​−N)+​1{w~i​>t−s}dF(s)=∑i≤(Q0​−N)+​(G(t−w~i​)−G(t)).
  3. Proposition 2.1 (p. 8): for a non-idling path and t≥0t\ge0t≥0,
∑i=1A(t)(G(t−τi−wi)−G(t−τi))=∫0t(Q(t−s)−N)+ dF(s)−∑i=1(Q0−N)+(G(t−w~i)−G(t)).\sum_{i=1}^{A(t)}\big(G(t-\tau_i-w_i)-G(t-\tau_i)\big)=\int_0^t(Q(t-s)-N)^+\,dF(s)-\sum_{i=1}^{(Q_0-N)^+}\big(G(t-\tilde w_i)-G(t)\big).i=1∑A(t)​(G(t−τi​−wi​)−G(t−τi​))=∫0t​(Q(t−s)−N)+dF(s)−i=1∑(Q0​−N)+​(G(t−w~i​)−G(t)).
  1. The decomposition (2.7) (p. 7): Q(t)=I(t)+W0(t)+M2(t)+AG(t)+∑i≤(Q0−N)+(G(t−w~i)−G(t))+∑i≤A(t)(G(t−τi−wi)−G(t−τi))Q(t)=I(t)+W_0(t)+M_2(t)+A_G(t)+\sum_{i\le(Q_0-N)^+}\big(G(t-\tilde w_i)-G(t)\big)+\sum_{i\le A(t)}\big(G(t-\tau_i-w_i)-G(t-\tau_i)\big)Q(t)=I(t)+W0​(t)+M2​(t)+AG​(t)+∑i≤(Q0​−N)+​(G(t−w~i​)−G(t))+∑i≤A(t)​(G(t−τi​−wi​)−G(t−τi​)).

Significance

The result. Equation (2.8) is the paper's starting point for both of its limit theorems. Writing Q−N=x+∫0t(Q(t−s)−N)+dF(s)Q-N=x+\int_0^t(Q(t-s)-N)^+dF(s)Q−N=x+∫0t​(Q(t−s)−N)+dF(s) with x=I+W0+M2+AG−Nx=I+W_0+M_2+A_G-Nx=I+W0​+M2​+AG​−N shows that the number in system is the image of the infinite-server quantities under the regulator map of the paper's §3, the unique solution of z(t)=x(t)+∫0t(z(t−s)+a)+dB(s)z(t)=x(t)+\int_0^t(z(t-s)+a)^+dB(s)z(t)=x(t)+∫0t​(z(t−s)+a)+dB(s). The fluid limit (Theorem 4.1, p. 12) and the diffusion limit (Theorem 5.1, p. 23) then follow from limit theorems for the infinite-server terms and the continuity of that map. Without (2.8), the NNN-server queue has no closed equation in terms of quantities whose limits are known.

Formalizing it. The identity is proved in the paper on pp. 7–9; to our knowledge it has not been machine-checked. This mission produces a sample-path model of the G/GI/NG/GI/NG/GI/N queue (initial customers, arrivals, FCFS service order, waiting times) together with the decomposition of its number in system, reusable for other many-server results stated pathwise. The limit theorems 4.1 and 5.1 themselves are weak-convergence statements in the Skorohod space and are not part of this mission; the regulator map (Proposition 3.1) is the subject of a separate mission of this series.

Difficulty

The decomposition (2.7) is bookkeeping. The content is Proposition 2.1, which converts a sum over customers of tail differences into an integral, against FFF, of the number of waiting customers at earlier times. The step that needs care is the change of viewpoint from "customer iii's service time falls in a window of length wiw_iwi​" to "customer iii is waiting at time t−st-st−s", after which the sum over customers must be recognised, at each time t−st-st−s, as the waiting count of the non-idling identity, with the arrivals counted up to A(t−s)A(t-s)A(t−s) rather than A(t)A(t)A(t). A second point is the treatment of an atom of FFF at 000: the half-open windows (t−τi−wi, t−τi](t-\tau_i-w_i,\,t-\tau_i](t−τi​−wi​,t−τi​] and the closed integration range [0,t][0,t][0,t] must match, and G(x)=1G(x)=1G(x)=1 for x<0x<0x<0 is used whenever a customer is still waiting.

Formalization scope

  • Paths are data, not random. A sample path is a structure holding NNN, Q0Q_0Q0​ and the sequences η~,η,τ,w,w~:N→R\tilde\eta,\eta,\tau,w,\tilde w:\mathbb N\to\mathbb Rη~​,η,τ,w,w~:N→R, indexed from 111 as in the paper. The arrival times are primary and A(t)A(t)A(t) is their counting function; for a counting process with A(0−)=0A(0-)=0A(0−)=0 this is the same data as the paper's τi=inf⁡{t≥0:A(t)≥i}\tau_i=\inf\{t\ge0:A(t)\ge i\}τi​=inf{t≥0:A(t)≥i}. The encoding has infinitely many arrivals, finitely many in each bounded interval.
  • Distributions are probability measures μ\muμ (for FFF) and μ0\mu_0μ0​ (for F0F_0F0​) on R\mathbb RR with no mass on (−∞,0)(-\infty,0)(−∞,0), and G(x)=μ((x,∞))G(x)=\mu((x,\infty))G(x)=μ((x,∞)) for every real xxx. The paper's i.i.d. assumptions and the mean-111 assumption on FFF are not used by these pathwise identities and are omitted.
  • Integrals ∫0t⋅ dF(s)\int_0^t\cdot\,dF(s)∫0t​⋅dF(s) are over the closed interval [0,t][0,t][0,t], an atom of FFF at 000 included; AG(t)A_G(t)AG​(t), a Stieltjes integral against the counting measure of arrivals, is the finite sum ∑i≤A(t)G(t−τi)\sum_{i\le A(t)}G(t-\tau_i)∑i≤A(t)​G(t−τi​).
  • Non-idling is a hypothesis on the sample path, exactly the identity displayed above. No hypothesis mentions (2.8), Proposition 2.1 or an integral against FFF, and QQQ is not a free function: it is defined by (2.2). Integrability of s↦(Q(t−s)−N)+s\mapsto(Q(t-s)-N)^+s↦(Q(t−s)−N)+ is part of what must be proved, not assumed. The hypotheses are jointly satisfiable, so the goal is not vacuous.
  • Needed infrastructure: finite sums, set integrals of indicator functions against a finite measure on R\mathbb RR, and the identity G(a)−G(b)=μ((a,b])G(a)-G(b)=\mu((a,b])G(a)−G(b)=μ((a,b]). All of it is in Mathlib.

Related platform items: WhittEfficiency.IS.* (the M/G/∞M/G/\inftyM/G/∞ queue with Poisson arrivals, the infinite-server object behind AGA_GAG​) and PalmQueueing.Palm.swiss_army_formula (a pathwise queueing identity of Palm calculus) are the nearest queueing results; neither is the object of this mission.

Selected references

  • J. Reed, The G/GI/N queue in the Halfin–Whitt regime, Ann. Appl. Probab. 19(6), 2009, 2211–2269. arXiv:0912.2837v1. https://arxiv.org/abs/0912.2837 , https://doi.org/10.1214/09-AAP609
  • S. Halfin and W. Whitt, Heavy-traffic limits for queues with many exponential servers, Oper. Res. 29, 1981, 567–588. https://doi.org/10.1287/opre.29.3.567 (MR629195)
  • A. A. Puhalskii and M. I. Reiman, The multiclass GI/PH/N queue in the Halfin–Whitt regime, Adv. Appl. Probab. 32, 2000, 564–595. https://mathscinet.ams.org/mathscinet-getitem?mr=1778580
8 thms1 active userReviewed
Dynamic ProgrammingMarkov Chain·Captain: mikedeng1

On Finding Optimal Policies in Discrete Dynamic Programming with No Discounting 2: f Maximizes Gain and Then Bias Exactly When Its Averaged Finite-Horizon Returns Dominate Every Stationary Policy'sResearch Paper

Motivation

A Markov decision process with finitely many states and actions is usually run either with a discount factor β<1\beta<1β<1 or under the long-run average criterion. Without discounting, the total return over an infinite horizon is typically infinite, and the average return per period ignores everything that happens in any finite stretch of time: two policies with equal gain (average return per period) can differ by a fixed amount of income forever, and the average criterion cannot tell them apart. Finer, undiscounted criteria that break such ties are a standard topic of the MDP literature (Puterman, Markov Decision Processes, Ch. 10), and the paper of this mission is one of their starting points.

Timeline. Howard (1960) introduced policy iteration for the average-return problem. Blackwell (1962) expanded the discounted return of a stationary policy near β=1\beta=1β=1 as x(f)/(1−β)+y(f)+o(1)x(f)/(1-\beta)+y(f)+o(1)x(f)/(1−β)+y(f)+o(1), defined 1-optimal ("nearly optimal") policies, and showed that the stationary 1-optimal policies are exactly those that maximize the gain x(f)x(f)x(f) and then the bias y(f)y(f)y(f). Veinott (1966), the source of this mission, gave a finite algorithm for such policies and, in §5, a characterization of the same set by an undiscounted criterion: comparing the Cesàro averages of the finite-horizon total returns. Later work (Veinott 1969) developed the hierarchy of nnn-discount optimality criteria from this starting point.

Setting

There are finitely many states sss and a finite set of actions; in state sss, action aaa earns the income i(s,a)i(s,a)i(s,a) and moves the system to state s′s's′ with probability q(s′∣s,a)q(s'\mid s,a)q(s′∣s,a). A decision rule f∈Ff\in Ff∈F chooses an action f(s)f(s)f(s) in each state; r(f)r(f)r(f) is the vector with entries i(s,f(s))i(s,f(s))i(s,f(s)), and Q(f)Q(f)Q(f) is the Markov matrix with entries q(s′∣s,f(s))q(s'\mid s,f(s))q(s′∣s,f(s)). A policy is a sequence π=(f1,f2,… )\pi=(f_1,f_2,\dots)π=(f1​,f2​,…) of decision rules, Qn(π)=Q(f1)⋯Q(fn)Q_n(\pi)=Q(f_1)\cdots Q(f_n)Qn​(π)=Q(f1​)⋯Q(fn​) with Q0(π)=IQ_0(\pi)=IQ0​(π)=I, and f∞=(f,f,… )f^\infty=(f,f,\dots)f∞=(f,f,…) is the stationary policy that uses fff every period.

The limit matrix Q∗(f)=lim⁡N→∞N−1∑i=0N−1Q(f)iQ^*(f)=\lim_{N\to\infty}N^{-1}\sum_{i=0}^{N-1}Q(f)^iQ∗(f)=limN→∞​N−1∑i=0N−1​Q(f)i exists for every Markov matrix. The gain and bias of fff are

x(f)=Q∗(f) r(f),y(f)=H(f) r(f),H(f)=(I−Q(f)+Q∗(f))−1−Q∗(f).x(f)=Q^*(f)\,r(f),\qquad y(f)=H(f)\,r(f),\qquad H(f)=\bigl(I-Q(f)+Q^*(f)\bigr)^{-1}-Q^*(f).x(f)=Q∗(f)r(f),y(f)=H(f)r(f),H(f)=(I−Q(f)+Q∗(f))−1−Q∗(f).

Vectors are compared coordinatewise. The decision rules of maximal gain form

F′={f∈F:x(f)≥x(g) for all g∈F},F'=\{f\in F: x(f)\ge x(g)\text{ for all }g\in F\},F′={f∈F:x(f)≥x(g) for all g∈F},

and those of maximal bias among them form

F′′={f∈F′:y(f)≥y(g) for all g∈F′}.F''=\{f\in F': y(f)\ge y(g)\text{ for all }g\in F'\}.F′′={f∈F′:y(f)≥y(g) for all g∈F′}.

Finally, the nnn-period total expected return of a policy π\piπ, starting from each state, is

Vn(π)=∑i=0n−1Qi(π) r(fi+1).V^n(\pi)=\sum_{i=0}^{n-1}Q_i(\pi)\,r(f_{i+1}).Vn(π)=i=0∑n−1​Qi​(π)r(fi+1​).

Formalization targets

Goal: Theorem 7

For every f∈Ff\in Ff∈F,

f∈F′′  ⟺  lim⁡N→∞1N∑n=1N[Vn(f∞)−Vn(g∞)] ≥ 0for all g∈F,f\in F''\iff \lim_{N\to\infty}\frac1N\sum_{n=1}^{N}\bigl[V^n(f^\infty)-V^n(g^\infty)\bigr]\ \ge\ 0\quad\text{for all }g\in F,f∈F′′⟺N→∞lim​N1​n=1∑N​[Vn(f∞)−Vn(g∞)] ≥ 0for all g∈F,

where the inequality is coordinatewise and the limit is taken in [−∞,+∞][-\infty,+\infty][−∞,+∞].

Milestones

  1. Theorem 3 (Blackwell): Vβ(f∞)=x(f)/(1−β)+y(f)+ε(β,f)V_\beta(f^\infty)=x(f)/(1-\beta)+y(f)+\varepsilon(\beta,f)Vβ​(f∞)=x(f)/(1−β)+y(f)+ε(β,f) with ε(β,f)→0\varepsilon(\beta,f)\to0ε(β,f)→0 as β→1−\beta\to1^-β→1−, where x(f)x(f)x(f) and y(f)y(f)y(f) are the unique solutions of [I−Q(f)]x=0, Q∗(f)x=Q∗(f)r(f)[I-Q(f)]x=0,\ Q^*(f)x=Q^*(f)r(f)[I−Q(f)]x=0, Q∗(f)x=Q∗(f)r(f) and [I−Q(f)]y=r(f)−x(f), Q∗(f)y=0[I-Q(f)]y=r(f)-x(f),\ Q^*(f)y=0[I−Q(f)]y=r(f)−x(f), Q∗(f)y=0.
  2. The nnn-step identity (p. 1293): y(f)=Vn(f∞)−n x(f)+Q(f)ny(f)y(f)=V^n(f^\infty)-n\,x(f)+Q(f)^n y(f)y(f)=Vn(f∞)−nx(f)+Q(f)ny(f) for every nnn.
  3. (27): y(f)=lim⁡N→∞N−1∑n=1N[Vn(f∞)−n x(f)]y(f)=\lim_{N\to\infty}N^{-1}\sum_{n=1}^N[V^n(f^\infty)-n\,x(f)]y(f)=limN→∞​N−1∑n=1N​[Vn(f∞)−nx(f)].
  4. (28): N−1∑n=1NVn(f∞)=N+12x(f)+y(f)+σ(N,f)N^{-1}\sum_{n=1}^N V^n(f^\infty)=\tfrac{N+1}{2}x(f)+y(f)+\sigma(N,f)N−1∑n=1N​Vn(f∞)=2N+1​x(f)+y(f)+σ(N,f) with σ(N,f)→0\sigma(N,f)\to0σ(N,f)→0.
  5. Theorem 4 (Blackwell): F′′F''F′′ is nonempty and is exactly the set of fff for which f∞f^\inftyf∞ is 1-optimal.

Significance

The result. Theorem 7 gives the set F′′F''F′′, defined through the discount-factor expansion, an interpretation that involves no discounting at all: f∞f^\inftyf∞ maximizes gain and then bias exactly when, from every starting state, its Cesàro-averaged finite-horizon returns are in the limit at least those of every other stationary policy. Combined with Theorem 4 it identifies the stationary 1-optimal policies with the stationary policies that are optimal under this average-overtaking comparison. The paper records a consequence (an optimal policy in the average-overtaking sense (26) that is stationary is 1-optimal) and conjectures the converse; that remark and conjecture are not part of this mission.

Formalizing it. The results are proved in the paper (Theorems 3 and 4 are Blackwell's). None of them is formalized: the Blackwell expansion (Theorem 3) is an open item on Prove2Me, referenced here, and Theorem 4, the nnn-step identity, (27), (28) and Theorem 7 have no machine-checked proof. A complete development gives the first formal treatment of Cesàro-averaged finite-horizon returns of a finite MDP and their relation to gain and bias.

Difficulty

The paper calls Theorem 7 "an immediate consequence of the representation (28)". For the direction "the limit condition implies f∈F′′f\in F''f∈F′′" this is accurate: a state where x(f)<x(g)x(f)<x(g)x(f)<x(g) would drive the average to −∞-\infty−∞, so f∈F′f\in F'f∈F′, and then the constant terms give the bias comparison over F′F'F′. The converse is where the obvious argument stops. It must hold for every g∈Fg\in Fg∈F, including g∉F′g\notin F'g∈/F′; at a state where x(f)s=x(g)sx(f)_s=x(g)_sx(f)s​=x(g)s​ for such a ggg, (28) needs y(f)s≥y(g)sy(f)_s\ge y(g)_sy(f)s​≥y(g)s​, and the definition of F′′F''F′′ only compares biases with gain-maximal rules. That inequality comes from 1-optimality of f∞f^\inftyf∞ (Theorem 4) together with the expansion of Theorem 3, not from (28). Proving (27) itself requires the Cesàro convergence N−1∑n<NQ(f)n→Q∗(f)N^{-1}\sum_{n<N}Q(f)^n\to Q^*(f)N−1∑n<N​Q(f)n→Q∗(f) and the identity Q∗(f)y(f)=0Q^*(f)y(f)=0Q∗(f)y(f)=0, which rest on Blackwell's Lemma 1 about Markov matrices.

Formalization scope

The development builds on the published Lean model of Blackwell (1962): Model (states St, actions Act, incomes, transition law), Policy (indexed from 000, so π 0 is f1f_1f1​), Qn, V, IsNearlyOptimal (= 1-optimal), limitMatrix, and the closed forms x, y. Conventions:

  • State and action sets are finite and nonempty, and every action is available in every state (As=AA_s=AAs​=A).
  • Vectors are real functions on states with the pointwise order; limits of vectors are coordinatewise; limits in NNN are along the natural numbers. The Cesàro mean in limitMatrix uses N+1N+1N+1 terms rather than Veinott's NNN; the limit is the same.
  • x(f)x(f)x(f) and y(f)y(f)y(f) are their closed forms; Theorem 3 asserts they are the unique solutions of Veinott's (2) and (3).
  • 1-optimality is Blackwell's "nearly optimal", formulated without U(β)U(\beta)U(β), against all policies.
  • In Theorem 7 the limit is an extended-real limit, stated as: for every ggg and every state, the real sequence converges in [−∞,+∞][-\infty,+\infty][−∞,+∞] to some L≥0L\ge0L≥0. A real-valued limit would make the statement false whenever x(f)s≠x(g)sx(f)_s\ne x(g)_sx(f)s​=x(g)s​; the comparison class is the stationary policies g∞g^\inftyg∞ only, and it must not be narrowed to g∈F′g\in F'g∈F′.

The shared module defines F′F'F′ and F′′F''F′′; this chunk defines Vn(π)V^n(\pi)Vn(π). Contributions are welcome for any milestone, and in particular for reusable lemmas about Cesàro means of powers of a Markov matrix, which the proofs of (27) and (28) and Blackwell's Lemma 1 share.

Selected references

  • A. F. Veinott, Jr., On Finding Optimal Policies in Discrete Dynamic Programming with No Discounting, Ann. Math. Statist. 37(5):1284–1294, 1966. https://doi.org/10.1214/aoms/1177699272
  • D. Blackwell, Discrete Dynamic Programming, Ann. Math. Statist. 33(2):719–726, 1962. https://doi.org/10.1214/aoms/1177704593
  • R. A. Howard, Dynamic Programming and Markov Processes, Wiley, New York, 1960.
  • A. F. Veinott, Jr., Discrete Dynamic Programming with Sensitive Discount Optimality Criteria, Ann. Math. Statist. 40(5):1635–1660, 1969. https://doi.org/10.1214/aoms/1177697379
  • M. L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming, Wiley, 1994. https://doi.org/10.1002/9780470316887
11 thms1 active userReviewed
Dynamic ProgrammingProbability·Captain: mikedeng1

Optimal Inventory Policies for Assembly Systems Under Random Demands 2: If an Item and Its Predecessors Have Negative Discounted Echelon Holding Cost, the Minimal Cost Is Unbounded BelowResearch Paper

Motivation

Assembly systems — components purchased from outside, assembled into subassemblies, and those into an end product facing random customer demand — are the standard model of material requirements planning under uncertainty. Rosling's paper (Oper. Res. 37(4), 1989) shows that, under a cost assumption, the optimal policies of such a system are those of an equivalent series system, so that the Clark–Scarf decomposition of serial systems (Clark and Scarf, 1960) applies to assembly systems.

The cost assumption of the main results requires every echelon holding cost to be positive. In §4 Rosling replaces it by a weaker Generalized Assumption (GA), which allows some echelon holding costs to be negative, and then argues with Theorem 4 that GA "covers all cases of practical interest for a long-run analysis". Part (i) of Theorem 4 is the first step of that argument: when GA(i) fails strictly for some item, the model is degenerate, because its minimal cost is −∞-\infty−∞. This mission formalizes that statement.

Setting

There are N≥1N \ge 1N≥1 items 1,…,N1, \dots, N1,…,N; item 111 is the end item. Every item i≥2i \ge 2i≥2 has exactly one immediate successor s(i)s(i)s(i) with 1≤s(i)<i1 \le s(i) < i1≤s(i)<i, and s(1)=0s(1) = 0s(1)=0, so the items form a tree rooted at the end item. Write A(i)A(i)A(i) for the set of all successors of iii, B(i)B(i)B(i) for the set of all its predecessors, and P(i)P(i)P(i) for its immediate predecessors. Item iii has a lead time li∈Nl_i \in \mathbb Nli​∈N, and its total lead time is M0=0M_0 = 0M0​=0, Mi=li+∑k∈A(i)lkM_i = l_i + \sum_{k\in A(i)} l_kMi​=li​+∑k∈A(i)​lk​; the items are indexed so that Mi−1≤MiM_{i-1} \le M_iMi−1​≤Mi​.

Time runs in periods t=1,2,…t = 1, 2, \dotst=1,2,…. The demands ξ1,ξ2,…\xi_1, \xi_2, \dotsξ1​,ξ2​,… for the end item are independent, identically distributed, nonnegative, with a density and a finite mean λ>0\lambda > 0λ>0. In period ttt a policy chooses the echelon inventory positions after ordering Y1t,…,YNtY_{1t}, \dots, Y_{Nt}Y1t​,…,YNt​ as functions of the demands already observed. The position before ordering is Xit=Yi,t−1−ξt−1X_{it} = Y_{i,t-1} - \xi_{t-1}Xit​=Yi,t−1​−ξt−1​, and the echelon stock on hand after arrivals is Xitl=Yi,t−li−∑r=t−lit−1ξrX^l_{it} = Y_{i,t-l_i} - \sum_{r=t-l_i}^{t-1}\xi_rXitl​=Yi,t−li​​−∑r=t−li​t−1​ξr​; positions referring to periods before 111 are read off given initial data x0x^0x0. Problem P asks for a policy satisfying

Xit≤Yit≤Xktlfor all k∈P(i) and all i,t(3)X_{it} \le Y_{it} \le X^l_{kt}\qquad\text{for all } k\in P(i) \text{ and all } i,t \tag{3}Xit​≤Yit​≤Xktl​for all k∈P(i) and all i,t(3)

that minimizes

E{∑t=1∞αt−1(∑i=1NαlihiYit+αl1(p+H1)∫Y1t∞(ξ−Y1t) φ1l+1(ξ) dξ)}+constant,(2)E\Big\{\sum_{t=1}^\infty \alpha^{t-1}\Big(\sum_{i=1}^N \alpha^{l_i} h_i Y_{it} + \alpha^{l_1}(p+H_1)\int_{Y_{1t}}^\infty(\xi - Y_{1t})\,\varphi_1^{l+1}(\xi)\,d\xi\Big)\Big\} + \text{constant}, \tag{2}E{t=1∑∞​αt−1(i=1∑N​αli​hi​Yit​+αl1​(p+H1​)∫Y1t​∞​(ξ−Y1t​)φ1l+1​(ξ)dξ)}+constant,(2)

where hih_ihi​ is the echelon holding cost of item iii, ppp the backlogging cost, H1H_1H1​ the installation holding cost of the end item, α\alphaα the discount factor and φ1l+1\varphi_1^{l+1}φ1l+1​ the density of the demand over l1+1l_1 + 1l1​+1 periods.

The quantity of interest is the discounted echelon holding cost of item iii and its predecessors,

hi α−Ms(i)+∑k∈B(i)hk α−Ms(k).h_i\,\alpha^{-M_{s(i)}} + \sum_{k\in B(i)} h_k\,\alpha^{-M_{s(k)}}.hi​α−Ms(i)​+k∈B(i)∑​hk​α−Ms(k)​.

GA(i) asks it to be positive for every item.

Formalization targets

Goal: Theorem 4(i), p. 574

If for some item iii

hi α−Ms(i)+∑k∈B(i)hk α−Ms(k)<0,h_i\,\alpha^{-M_{s(i)}} + \sum_{k\in B(i)} h_k\,\alpha^{-M_{s(k)}} < 0,hi​α−Ms(i)​+k∈B(i)∑​hk​α−Ms(k)​<0,

then the minimal cost of Problem P is unbounded below: for every real CCC there is a feasible policy whose cost is a real number less than CCC.

Milestones: the proof of Theorem 4(i), p. 578

  1. The one-more-unit policy is feasible. Ordering extra units of every item kkk of the subsystem {i}∪B(i)\{i\}\cup B(i){i}∪B(i) in period Mm−Mk+1M_m - M_k + 1Mm​−Mk​+1, where MmM_mMm​ is the greatest total lead time in the subsystem, and holding them forever, preserves (3).
  2. The total cost increase. For a non-end item iii and a policy of finite cost, δ\deltaδ extra units change the cost by exactly
δ αMm−Mi αli(hi+∑k∈B(i)hk α−(Ms(k)−Ms(i)))1−α.\delta\,\alpha^{M_m - M_i}\,\frac{\alpha^{l_i}\big(h_i + \sum_{k\in B(i)} h_k\,\alpha^{-(M_{s(k)} - M_{s(i)})}\big)}{1-\alpha}.δαMm​−Mi​1−ααli​(hi​+∑k∈B(i)​hk​α−(Ms(k)​−Ms(i)​))​.

Significance

Theorem 4(i) explains why the Generalized Assumption is the natural boundary of the theory: a strict violation of GA(i) makes Problem P meaningless, since no policy is optimal and the infimum is −∞-\infty−∞. Together with parts (ii) and (iii) of Theorem 4 it reduces every case of interest to systems satisfying GA, for which Rosling's series-system results apply. The quantity in the condition is the natural "net value of stockpiling" of a subsystem, and the same exchange — buy more of a subsystem early and hold it forever — is the basic perturbation behind many optimality arguments for multi-echelon systems.

The result is proved in the paper, in a few lines. To our knowledge neither Theorem 4 nor the assembly model of Problem P has been machine-checked. A formalization adds a precise statement of what "the minimal cost is unbounded below" means for a stochastic infinite-horizon problem with an extended-real cost, and the definitions of the assembly model — product tree, total lead times, echelon positions, history-dependent policies, constraint (3) and objective (2) — which other statements of the same paper need.

Difficulty

The algebra of the cost increase is a geometric series. The work lies elsewhere. First, the one-more-unit policy must be shown feasible for every item of the subsystem in every period, including the boundary items: the successor of iii, whose constraint loosens, and predecessors with zero lead time. Second, the cost of a policy is an expectation of an infinite discounted sum whose terms have no sign; the cost increase can only be added to a cost that is finite, and the argument needs a feasible policy of finite cost to start from, which the paper takes for granted. Third, for the end item i=1i = 1i=1 the extra units also change the expected backlog term of (2), which the printed display omits, so the end item needs a separate bound.

Formalization scope

  • Periods start at Lean index k=0k = 0k=0 with k=t−1k = t - 1k=t−1; the discount weight αt−1\alpha^{t-1}αt−1 is αk\alpha^kαk, and coordinate jjj of a demand path is ξj+1\xi_{j+1}ξj+1​. Items are natural numbers 1..N1..N1..N.
  • Demand is the product law (Measure.infinitePi) of a probability measure ν\nuν on R\mathbb RR with ν((−∞,0))=0\nu((-\infty,0)) = 0ν((−∞,0))=0, ν≪\nu \llν≪ Lebesgue, ν\nuν integrable and ∫x dν>0\int x\,d\nu > 0∫xdν>0.
  • Policies are history-dependent and measurable; feasibility (3) holds almost surely.
  • Initial data xi0(s)x^0_i(s)xi0​(s), s≥1s \ge 1s≥1, the echelon position of item iii at the start of period 1 ordered sss periods ago or earlier, are assumed well formed: nonincreasing in sss, and xi0(s)≤xk0(s+lk)x^0_i(s) \le x^0_k(s + l_k)xi0​(s)≤xk0​(s+lk​) for k∈P(i)k \in P(i)k∈P(i). The paper takes this for granted; it is what makes Problem P feasible.
  • Cost is (2) without its policy-independent constant, as an extended real E[∑(αt−1ct)+]−E[∑(αt−1ct)−]E[\sum(\alpha^{t-1}c_t)^+] - E[\sum(\alpha^{t-1}c_t)^-]E[∑(αt−1ct​)+]−E[∑(αt−1ct​)−].
  • Added restriction: 0<α<10 < \alpha < 10<α<1. The page allows α=1\alpha = 1α=1, where the average cost is minimized and is defined only through a limit recipe.
  • No sign condition is imposed on hih_ihi​, ppp or H1H_1H1​; neither the Assumption nor GA is a hypothesis.

In the extended reals, ⊤−⊤=⊥\top - \top = \bot⊤−⊤=⊥: a policy whose positive and negative cost parts are both infinite gets the value −∞-\infty−∞. The goal therefore asks for policies of real cost below every bound; a formalization that only exhibits a policy of cost ⊥\bot⊥ would be trivially true and is ruled out. Milestone 2 is stated for i≥2i \ge 2i≥2, where the backlog term is unchanged.

Needed infrastructure: lower Lebesgue integrals of discounted sums under the infinite product measure, linearity of the cost under a deterministic perturbation, and the finiteness of the cost of the "never order" policy (whose decisions decrease linearly in the accumulated demand). Contributions of general lemmas on discounted costs of history-dependent policies under infinitePi are welcome and reusable.

Selected references

  • K. Rosling, Optimal Inventory Policies for Assembly Systems under Random Demands, Operations Research 37(4):565–579, 1989. https://doi.org/10.1287/opre.37.4.565
  • A. J. Clark and H. Scarf, Optimal Policies for a Multi-Echelon Inventory Problem, Management Science 6(4):475–490, 1960. https://doi.org/10.1287/mnsc.6.4.475
  • A. Federgruen and P. Zipkin, Computational Issues in an Infinite-Horizon, Multiechelon Inventory Model, Operations Research 32(4):818–836, 1984. https://doi.org/10.1287/opre.32.4.818
6 thms1 active userReviewed
Dynamic ProgrammingMarkov ChainOptimization·Captain: mikedeng1

Inventory Control in a Fluctuating Demand Environment II: With a Fixed Order Cost, a World-Dependent (r, S) Policy Is Optimal and Lies within Veinott-Type BoundsResearch Paper

Motivation

Demand for many products is not stationary: it rises and falls with the economy, the season, a product's life cycle or a customer's state. Song and Zipkin (1993) (DOI 10.1287/opre.41.2.351) model this by a world-driven demand process: an exogenous Markov chain AAA (the "state of the world") whose current state sets the rate of a Poisson demand stream. The model contains the classical stationary inventory model as the case of one world state, and it is a standard example of a Markov-modulated decision problem in operations research.

For stationary demand with a fixed order cost, optimal (s,S)(s, S)(s,S) policies go back to Scarf (1960) for finite horizons and Iglehart (1963) for the infinite horizon; Veinott (1966) bounded the optimal parameters by quantities computed from the one-period cost alone. This mission formalizes the extension of both results to the fluctuating-demand model (§3.2 of the paper). A companion mission treats the linear order-cost case, in which a world-dependent basestock policy is optimal.

Setting

The world AAA is a continuous-time Markov chain on a countable set III with generator (qij)(q_{ij})(qij​), qi=−qii=∑j≠iqijq_i = -q_{ii} = \sum_{j\neq i} q_{ij}qi​=−qii​=∑j=i​qij​, and bounded rates. While A=iA = iA=i, unit demands arrive at Poisson rate λi\lambda_iλi​; shortages are backlogged. An order arrives after a random lead time LLL, independent of the world and the demand. The inventory position x∈Zx \in \mathbb Zx∈Z is on-hand stock minus backorders plus stock on order.

Costs: a fixed cost Kˉ\bar KKˉ per order and a unit cost cˉ\bar ccˉ, both paid on arrival; a holding cost rate h>0h > 0h>0 and a penalty cost rate p>0p > 0p>0; a discount rate α>0\alpha > 0α>0. With F~L(α)=E[e−αL]\tilde F_L(\alpha) = E[e^{-\alpha L}]F~L​(α)=E[e−αL], the discounted costs are c=cˉF~L(α)c = \bar c\tilde F_L(\alpha)c=cˉF~L​(α) and K=KˉF~L(α)K = \bar K\tilde F_L(\alpha)K=KˉF~L​(α). If DLiD^i_LDLi​ is the demand during a lead time from world state iii and C^(x)=max⁡{−px,hx}\hat C(x) = \max\{-px, hx\}C^(x)=max{−px,hx},

C(i,y)=E[e−αLC^(y−DLi)].C(i, y) = E\big[e^{-\alpha L}\hat C(y - D^i_L)\big].C(i,y)=E[e−αLC^(y−DLi​)].

Uniformization at a rate μ≥sup⁡iqi+sup⁡iλi\mu \ge \sup_i q_i + \sup_i\lambda_iμ≥supi​qi​+supi​λi​ turns the problem into a discrete-time dynamic program with β=1/(μ+α)\beta = 1/(\mu+\alpha)β=1/(μ+α), γ=βμ\gamma = \beta\muγ=βμ and the myopic cost G+(i,y)=(1−γ)cy+βC(i,y)G^+(i, y) = (1-\gamma)cy + \beta C(i, y)G+(i,y)=(1−γ)cy+βC(i,y). For a terminal cost W0W_0W0​ the nnn-stage costs satisfy the recursion (10):

Wn(i,x)=min⁡y≥x{Kδ(y−x)+Gn(i,y)},W_n(i, x) = \min_{y\ge x}\{K\delta(y-x) + G_n(i, y)\},Wn​(i,x)=y≥xmin​{Kδ(y−x)+Gn​(i,y)}, Gn(i,y)=G+(i,y)+βλic+β{λiWn−1(i,y−1)+∑j≠iqijWn−1(j,y)+(μ−λi−qi)Wn−1(i,y)},G_n(i, y) = G^+(i, y) + \beta\lambda_i c + \beta\Big\{\lambda_i W_{n-1}(i, y-1) + \sum_{j\neq i} q_{ij}W_{n-1}(j, y) + (\mu - \lambda_i - q_i)W_{n-1}(i, y)\Big\},Gn​(i,y)=G+(i,y)+βλi​c+β{λi​Wn−1​(i,y−1)+j=i∑​qij​Wn−1​(j,y)+(μ−λi​−qi​)Wn−1​(i,y)},

where δ(z)=1\delta(z) = 1δ(z)=1 if z>0z > 0z>0 and δ(0)=0\delta(0) = 0δ(0)=0. Throughout, Assumption 1, αcˉ<p\alpha\bar c < pαcˉ<p, holds and K>0K > 0K>0. The terminal cost is the optimal cost W∞W_\inftyW∞​ of the linear model (K=0K = 0K=0); its G∞G_\inftyG∞​ is written G0G_0G0​, and y∗(i)y^*(i)y∗(i) is the smallest minimizer of G0(i,⋅)G_0(i, \cdot)G0​(i,⋅).

A function f:Z→Rf : \mathbb Z \to \mathbb Rf:Z→R is KKK-convex (Definition 2) if f(x)−f(x−b)ba+f(x)≤f(x+a)+K\frac{f(x)-f(x-b)}{b}a + f(x) \le f(x+a) + Kbf(x)−f(x−b)​a+f(x)≤f(x+a)+K for all xxx and all integers a,b>0a, b > 0a,b>0. The (r,S)(r, S)(r,S) policy with parameters {(r(i),S(i))}\{(r(i), S(i))\}{(r(i),S(i))} orders up to S(i)S(i)S(i) when x≤r(i)x \le r(i)x≤r(i) and the world is in state iii, and does not order otherwise. With y+(i)y^+(i)y+(i) the smallest minimizer of G+(i,⋅)G^+(i,\cdot)G+(i,⋅) and ymin⁡+=min⁡iy+(i)y^+_{\min} = \min_i y^+(i)ymin+​=mini​y+(i), the paper defines

S+(i)=min⁡{y≥y+(i):G+(i,y)−G+(i,y+(i))>γK},r+(i)=max⁡{y<y+(i):G+(i,y)−G+(i,y+(i))>(1−γ)K},r−(i)=max⁡{y<y∗(i):G0(i,y)−G0(i,y∗(i))>K},r−−(i)=max⁡{y<ymin⁡+:G+(i,y)−G+(i,ymin⁡+)>K}.\begin{aligned} S^+(i) &= \min\{y \ge y^+(i) : G^+(i, y) - G^+(i, y^+(i)) > \gamma K\},\\ r^+(i) &= \max\{y < y^+(i) : G^+(i, y) - G^+(i, y^+(i)) > (1-\gamma)K\},\\ r^-(i) &= \max\{y < y^*(i) : G_0(i, y) - G_0(i, y^*(i)) > K\},\\ r^{--}(i) &= \max\{y < y^+_{\min} : G^+(i, y) - G^+(i, y^+_{\min}) > K\}. \end{aligned}S+(i)r+(i)r−(i)r−−(i)​=min{y≥y+(i):G+(i,y)−G+(i,y+(i))>γK},=max{y<y+(i):G+(i,y)−G+(i,y+(i))>(1−γ)K},=max{y<y∗(i):G0​(i,y)−G0​(i,y∗(i))>K},=max{y<ymin+​:G+(i,y)−G+(i,ymin+​)>K}.​

Formalization targets

Goal: Theorems 5(e) and 6

Let Sn∗(i)S^*_n(i)Sn∗​(i) be the smallest minimizer of Gn(i,⋅)G_n(i,\cdot)Gn​(i,⋅) and rn∗(i)r^*_n(i)rn∗​(i) the largest y<Sn∗(i)y < S^*_n(i)y<Sn∗​(i) with Gn(i,y)>K+Gn(i,Sn∗(i))G_n(i, y) > K + G_n(i, S^*_n(i))Gn​(i,y)>K+Gn​(i,Sn∗​(i)). For every iii the sequence (rn∗(i),Sn∗(i))(r^*_n(i), S^*_n(i))(rn∗​(i),Sn∗​(i)) has limit points; for any choice of limit points (r∗(i),S∗(i))(r^*(i), S^*(i))(r∗(i),S∗(i)), the world-dependent (r,S)(r, S)(r,S) policy with these parameters is optimal for the infinite-horizon discounted problem, and

y∗(i)≤S∗(i)<S+(i),r−−(i)≤r−(i)≤r∗(i)≤r+(i).y^*(i) \le S^*(i) < S^+(i),\qquad r^{--}(i) \le r^-(i) \le r^*(i) \le r^+(i).y∗(i)≤S∗(i)<S+(i),r−−(i)≤r−(i)≤r∗(i)≤r+(i).

Milestones

Lemma 4 (uniform bounds on the iterates); Theorem 3 (KKK-convexity of GnG_nGn​, WnW_nWn​ and optimality of an (r,S)(r, S)(r,S) rule for each nnn-stage problem); Lemmas 5 and 6 (difference comparisons); Theorem 4 (the bounds above for every nnn); Theorem 5(a)–(d) (convergence of WnW_nWn​, GnG_nGn​, the (r,S)(r, S)(r,S) property of limit points, and the optimality equation (2)).

Significance

The result shows that the classical (s,S)(s, S)(s,S) structure survives Markov modulation of demand with an infinite world space and stochastic lead times: a stationary policy whose two parameters depend only on the current world state is optimal over the infinite horizon. The bounds of Theorem 6 locate those parameters in an interval computed from the myopic cost G+G^+G+ and the linear model alone, uniformly in the horizon.

The results are proved in the paper, except that Theorem 5 refers to Iglehart (1963) and Lemma 4 to Song's thesis. To our knowledge none of them has a machine-checked proof. A formalization would also produce reusable machinery: KKK-convexity on the integers, the Scarf-type inductive argument with a countable world state, and the passage from finite-horizon to infinite-horizon optimality for a nonnegative-cost Markov decision process with unbounded costs.

Difficulty

The one-period cost is unbounded in xxx and the world space may be infinite, so the usual contraction argument for discounted dynamic programs does not apply. Convergence of WnW_nWn​ needs the uniform bound of Lemma 4. Optimality of the limit policy needs a separate argument, because a pointwise limit of (r,S)(r, S)(r,S) rules need not exist: the parameter sequences are only known to have limit points. The lower bounds r−r^-r− and r−−r^{--}r−− hold only because the terminal cost is the linear model's optimal cost, which makes the difference comparisons of Lemma 6 possible; the paper notes that Lemma 6 depends on this choice of W0W_0W0​.

Formalization scope

All definitions live in the namespace SongZipkinFluct.FixedCost. Inventory positions and order-up-to levels are integers. The demand law DLiD^i_LDLi​ is defined through uniformization at rate μ\muμ, as an exact series representation of the Markov-modulated Poisson count, and the lead time is a probability law on [0,∞)[0,\infty)[0,∞) independent of the rest. The standing conventions h,p,α>0h, p, \alpha > 0h,p,α>0, cˉ,Kˉ≥0\bar c, \bar K \ge 0cˉ,Kˉ≥0, μ>0\mu > 0μ>0 and μ≥q∗+λ∗\mu \ge q^* + \lambda^*μ≥q∗+λ∗ are hypotheses of the model. Assumption 1 and Kˉ>0\bar K > 0Kˉ>0 are hypotheses of each theorem.

The iterates WnW_nWn​ are defined by the recursion (10), with the minimum written as an infimum over integers y≥xy \ge xy≥x. W∞W_\inftyW∞​ and G∞G_\inftyG∞​ are the suprema over nnn, which are finite and equal the limits by Lemma 4 and Theorem 3. Every minimizer and every extremal integer (y+y^+y+, ymin⁡+y^+_{\min}ymin+​, y∗y^*y∗, Sn∗S^*_nSn∗​, rn∗r^*_nrn∗​, S+S^+S+, r+r^+r+, r−r^-r−, r−−r^{--}r−−) is passed through its defining property, never through sInf/sSup on Z\mathbb ZZ. The goal also asserts that all of them exist, so its hypotheses can be met. A limit point of an integer sequence is a value taken infinitely often.

Optimality is over all deterministic history-dependent feasible policies and from every initial state. Costs are valued in [0,∞][0,\infty][0,∞], and a policy's cost is the supremum of its finite-horizon discounted costs in the uniformized problem (1). Restricting the comparison to (r,S)(r, S)(r,S) or stationary policies would trivialize the goal and is ruled out. KKK-convexity is the integer version of Definition 2, not convexity over Z\mathbb ZZ-scalars.

Welcome contributions: KKK-convexity calculus on Z\mathbb ZZ, convergence of the uniformized demand series, Lemma 4, and an Iglehart-type limit argument for countable-state Markov decision processes with nonnegative costs.

Selected references

  • J.-S. Song and P. Zipkin, Inventory Control in a Fluctuating Demand Environment, Operations Research 41(2):351–370, 1993. https://doi.org/10.1287/opre.41.2.351
  • H. Scarf, The Optimality of (S, s) Policies in the Dynamic Inventory Problem, in Mathematical Methods in the Social Sciences, Stanford University Press, 1960.
  • D. L. Iglehart, Optimality of (s, S) Policies in the Infinite Horizon Dynamic Inventory Problem, Management Science 9(2):259–267, 1963. https://doi.org/10.1287/mnsc.9.2.259
  • A. F. Veinott, Jr., On the Optimality of (s, S) Inventory Policies: New Conditions and a New Proof, SIAM Journal on Applied Mathematics 14(5):1067–1083, 1966. https://doi.org/10.1137/0114086
  • D. P. Bertsekas and S. E. Shreve, Stochastic Optimal Control: The Discrete-Time Case, Academic Press, 1978.
14 thms1 active userReviewed
Dynamic ProgrammingProbability·Captain: mikedeng1

Optimal Ordering and Rationing Policies in a Nonstationary Dynamic Inventory Model with n Demand Classes III: Under Full Backlogging, Class-j Rationing Levels Ignore Lower-Class Penalties and DemandsResearch Paper

Motivation

A single stock of one product often serves customers of different importance: emergency and routine orders for spare parts, contract and spot customers, high- and low-priority patients for a blood bank. When stock runs low, the question is not only how much to order but how much of the remaining stock to hand to the less important customers now, and how much to hold back for more important demand that may still arrive. Stock rationing policies answer this question with critical rationing levels: demand of a class is served only while the stock stays above that class's level.

Topkis (1968) studied this problem as a finite-horizon dynamic program with nnn demand classes, nonstationary costs and demands, and an arbitrary degree of backlogging. His Theorem 1 shows that, when each interval has either complete backlogging or none, a policy given by critical levels zˉt1≥⋯≥zˉtn\bar z_t^1 \ge \dots \ge \bar z_t^nzˉt1​≥⋯≥zˉtn​ is optimal. This mission concerns his Theorem 3, which treats the case of complete backlogging: the critical levels of a class do not depend on what happens in the classes below it.

Timeline. Veinott (1965) considered nnn demand classes but imposed the policy with all critical levels equal to 000. Topkis's 1966 report treated n=2n = 2n=2, partially duplicated independently by Evans (1968) and by Kaplan's 1966 report "Stock Rationing", in which, according to Topkis, the result of Theorem 3 was used implicitly. Topkis (1968) extended the analysis to nnn classes and stated Theorem 3 explicitly.

Setting

A period is divided into kkk intervals, indexed backwards: interval ttt is followed by t−1t - 1t−1 further intervals, and interval kkk is the first. There are nnn demand classes; class nnn is the most important. In interval ttt a random demand vector dt=(dt1,…,dtn)≥0d_t = (d_t^1, \dots, d_t^n) \ge 0dt​=(dt1​,…,dtn​)≥0 with law μt\mu_tμt​ and finite means arrives; demands in different intervals are independent. Given the stock zzz and the outstanding demand B=b+dtB = b + d_tB=b+dt​ (backlog bbb plus new demand), the decision is the vector uuu, 0≤u≤B0 \le u \le B0≤u≤B, of demand left unsatisfied; the stock drops to w=z−1⋅(B−u)≥0w = z - \mathbf 1 \cdot (B - u) \ge 0w=z−1⋅(B−u)≥0. A penalty pt⋅up_t \cdot upt​⋅u and a holding cost ht(w)h_t(w)ht​(w) are charged, and the backlog carried into the next interval is atua_t uat​u, with at=1a_t = 1at​=1 for complete backlogging. At the end of the period a salvage cost g0(z,b)=v1(z)+v2(z−1⋅b)g_0(z, b) = v_1(z) + v_2(z - \mathbf 1 \cdot b)g0​(z,b)=v1​(z)+v2​(z−1⋅b) is charged.

The standing assumptions are: hth_tht​ convex and continuous on [0,∞)[0,\infty)[0,∞); v1v_1v1​ convex and continuous on [0,∞)[0,\infty)[0,∞); v2v_2v2​ convex and continuous on R\mathbb RR with lim⁡w→−∞D+v2(w)>−∞\lim_{w \to -\infty} D^+ v_2(w) > -\inftylimw→−∞​D+v2​(w)>−∞; and 0≤pt1≤⋯≤ptn0 \le p_t^1 \le \dots \le p_t^n0≤pt1​≤⋯≤ptn​. The minimal expected cost satisfies the recursion (1):

ft(z,B)=inf⁡0≤u≤Bw=z−1⋅(B−u)≥0[pt⋅u+ht(w)+gt−1(w,atu)],gt(z,b)=E ft(z,b+dt).f_t(z, B) = \inf_{\substack{0 \le u \le B \\ w = z - \mathbf 1\cdot(B-u) \ge 0}} \big[p_t \cdot u + h_t(w) + g_{t-1}(w, a_t u)\big], \qquad g_t(z, b) = \mathbb E\, f_t(z, b + d_t).ft​(z,B)=0≤u≤Bw=z−1⋅(B−u)≥0​inf​[pt​⋅u+ht​(w)+gt−1​(w,at​u)],gt​(z,b)=Eft​(z,b+dt​).

With δj\delta_jδj​ the jjj-th unit vector, the critical rationing level zˉtj\bar z_t^jzˉtj​ is +∞+\infty+∞ if w↦ptjw+ht(w)+gt−1(w,atwδj)w \mapsto p_t^j w + h_t(w) + g_{t-1}(w, a_t w \delta_j)w↦ptj​w+ht​(w)+gt−1​(w,at​wδj​) is strictly decreasing on [0,∞)[0, \infty)[0,∞), and is the smallest minimizer of that function on [0,∞)[0,\infty)[0,∞) otherwise. D+D^+D+ denotes a right derivative.

Formalization targets

Goal: Theorem 3

Assume at+1=at=⋯=a1=1a_{t+1} = a_t = \dots = a_1 = 1at+1​=at​=⋯=a1​=1 and that the demands of different classes are independent in each interval. Fix a class jjj. Then for two instances of the model that differ only in the penalties pimp_i^mpim​ of the classes m≤j−1m \le j - 1m≤j−1 and in the demand distributions of the classes m≤jm \le jm≤j,

(a) the marginal quantities

Dz+gt(z,zδj)andDε+gt(w,ε(δj−δs)+b)∣ε=0(s>j, bs>0)D_z^+ g_t(z, z\delta_j) \quad\text{and}\quad D_\varepsilon^+ g_t\big(w, \varepsilon(\delta_j - \delta_s) + b\big)\big|_{\varepsilon = 0} \quad (s > j,\ b^s > 0)Dz+​gt​(z,zδj​)andDε+​gt​(w,ε(δj​−δs​)+b)​ε=0​(s>j, bs>0)

coincide, and

(b) the critical rationing levels zˉt+1j\bar z_{t+1}^jzˉt+1j​ coincide.

Milestones

  1. The claim in the proof of Theorem 3 (p. 173): for j<sj < sj<s and Bs>0B^s > 0Bs>0, Dε+gt(z,ε(δj−δs)+B)∣ε=0D_\varepsilon^+ g_t(z, \varepsilon(\delta_j - \delta_s) + B)|_{\varepsilon=0}Dε+​gt​(z,ε(δj​−δs​)+B)∣ε=0​ is independent of B1,…,BjB^1, \dots, B^jB1,…,Bj.
  2. Theorem 3 (a) on its own, from which the page derives (b).

Significance

The result. Theorem 3 reduces the size of the problem. Under complete backlogging, class 1 demand does not influence any critical rationing level, and the class-jjj critical levels can be found from a problem with only the n−j+1n - j + 1n−j+1 classes j,…,nj, \dots, nj,…,n, with a degenerate demand distribution for class jjj. Since computing the levels requires tabulating functions of several variables, which Topkis calls prohibitive for large nnn, this is the observation that makes the levels of the most important classes computable.

Formalizing it. The theorem is proved in the paper by a short sketch that points to two case formulas, (16) and (17), to Theorem 1 (c), and to an interchange of expectation and right derivative. A machine-checked proof would supply the induction, the case analysis and the interchange in full. No formalization of any part of Topkis's paper is known: the result is proved on paper, not formalized.

Difficulty

The obvious argument is an induction on ttt showing that gtg_tgt​ itself does not depend on the lower classes. It fails: gtg_tgt​ does depend on the penalties and demands of every class, since lower-class demand is still served and penalized. Only certain directional derivatives are invariant, and these derivatives depend on the critical levels of every class l≥jl \ge jl≥j and on the optimal rationing decision, which themselves change from one instance to the other unless the invariance is already known for the previous interval. The derivatives are one-sided, may be −∞-\infty−∞ at the boundary, and must be passed through an expectation.

Formalization scope

The Lean development, in namespace TopkisRation.Backlog, fixes these conventions:

  • classes are Fin n (class jjj of the paper is index j−1j-1j−1); intervals are natural numbers counted backwards; vectors are ordered pointwise;
  • ftf_tft​ is defined by the recursion (1), with sInf for the infimum and the Bochner integral for E\mathbb EE, and every statement restricts to z≥0z \ge 0z≥0, b≥0b \ge 0b≥0;
  • D+D^+D+ is an extended-real right derivative (a liminf of difference quotients); Dz+gt(z,zδj)D_z^+ g_t(z, z\delta_j)Dz+​gt​(z,zδj​) is the derivative along the ray in which the stock and the class-jjj backlog move together;
  • zˉtj\bar z_t^jzˉtj​ takes values in R∪{+∞}\mathbb R \cup \{+\infty\}R∪{+∞} and is specified by a predicate (smallest minimizer, or +∞+\infty+∞ for a strictly decreasing function), never by a real sInf;
  • the limit in (B) is read as "D+v2D^+ v_2D+v2​ is bounded below";
  • independence of the classes in interval iii is independence of the coordinate maps under μi\mu_iμi​.

"Does not depend on" is stated with two instances MMM, M′M'M′ that agree on every datum the conclusion uses except the freed ones: the same kkk, v1v_1v1​, v2v_2v2​, the same hih_ihi​ and the same pimp_i^mpim​ for m≥jm \ge jm≥j in intervals i≤t+1i \le t+1i≤t+1, and the same class-mmm marginal law for m>jm > jm>j in every interval. The penalty of class jjj itself is not freed. Data of intervals beyond t+1t+1t+1 and the ordering cost are unconstrained. Part (b) is stated for arbitrary critical levels of the two instances; existence of critical levels is part of the paper's §1 and is not assumed away, so (b) is not vacuous. The goal does not assume the p. 173 claim, the case formulas (16)–(17), or Theorem 1. Encoding "does not depend" as "is a function of the remaining data" is ruled out.

Needed infrastructure: one-sided derivatives of convex functions and their monotone limits, interchange of right derivative and expectation for convex integrands, and the convexity and rationing-policy results of §1 (Theorem 1, Lemma 2). These are reusable for the other missions of this series. Contributions welcome: proofs of the milestones, a formal version of (16)–(17), and the existence of critical levels.

Selected references

  • D. M. Topkis, Optimal ordering and rationing policies in a nonstationary dynamic inventory model with n demand classes, Management Science 15(3):160–176, 1968. https://doi.org/10.1287/mnsc.15.3.160
  • A. F. Veinott Jr., Optimal policy in a dynamic, single product, nonstationary inventory model with several demand classes, Operations Research 13(5):761–778, 1965. https://doi.org/10.1287/opre.13.5.761
  • R. V. Evans, Sales and restocking policies in a single item inventory system, Management Science 14(7):463–472, 1968. https://doi.org/10.1287/mnsc.14.7.463
  • A. Kaplan, Stock Rationing, report, December 1966 (reference [4] of Topkis 1968).
5 thms1 active userReviewed
Dynamic ProgrammingProbability·Captain: mikedeng1

Optimal Ordering and Rationing Policies in a Nonstationary Dynamic Inventory Model with n Demand Classes II: Without Backlogging, the Critical Rationing Levels Are Nondecreasing in tResearch Paper

Motivation

A firm that holds a single stock of one product often faces demand from several classes of customers that differ in importance: emergency and routine orders for spare parts, contract and spot customers, high- and low-priority users of a military supply system. When stock runs low it can pay to refuse a less important demand now in order to keep stock for a more important demand that may arrive later. Veinott (Operations Research 13 (1965), 761–778) studied a dynamic inventory model with several demand classes but required the specific policy that serves every class whenever stock is available. Topkis (Management Science 15 (1968), 160–176) treated the dynamic version, in which demands arrive over the intervals of a period between two procurements, and showed that the optimal rationing policy has a simple form described by one critical rationing level per class and interval.

This mission concerns how those critical levels move over time. The paper's Theorem 2 gives a condition under which, without backlogging, the levels are monotone in the interval index, so that rationing is at least as strict early in the period as late in it. It is the second of four missions on the paper; the first establishes the critical-level policy itself (Theorem 1).

Setting

A period is divided into kkk intervals, indexed backwards: interval ttt has t−1t-1t−1 intervals after it, so interval kkk is the first and interval 1 the last. There are nnn demand classes, class nnn the most important. At the start of interval ttt the demand vector dt=(dt1,…,dtn)d_t = (d_t^1,\dots,d_t^n)dt​=(dt1​,…,dtn​) is observed; demands of different intervals are independent, and each class has a finite mean. With backlog bbb and stock zzz, one decides the vector uuu of outstanding demand left unsatisfied, 0≤u≤B=b+dt0 \le u \le B = b + d_t0≤u≤B=b+dt​, using 1⋅(B−u)\mathbf 1\cdot(B-u)1⋅(B−u) units of stock, so the stock at the end of the interval is w=z−1⋅(B−u)≥0w = z - \mathbf 1\cdot(B-u) \ge 0w=z−1⋅(B−u)≥0. A fraction at≥0a_t \ge 0at​≥0 of the unsatisfied demand is carried as backlog atua_t uat​u into the next interval: at=1a_t = 1at​=1 is complete backlogging, at=0a_t = 0at​=0 none.

The costs are a penalty pt⋅up_t\cdot upt​⋅u with 0≤pt1≤⋯≤ptn0 \le p_t^1 \le \dots \le p_t^n0≤pt1​≤⋯≤ptn​ (Assumption (C)), a holding cost ht(w)h_t(w)ht​(w) continuous and convex on [0,∞)[0,\infty)[0,∞) (Assumption (A)), and a salvage cost v1(z)+v2(z−1⋅b)v_1(z) + v_2(z - \mathbf 1\cdot b)v1​(z)+v2​(z−1⋅b) at the end of the period, with v1v_1v1​, v2v_2v2​ convex and continuous and D+v2D^+v_2D+v2​ bounded below (Assumption (B)). Here D+D^+D+ denotes the right derivative. The optimal costs satisfy the recursion (1)

ft(z,B)=inf⁡0≤u≤B, w=z−1⋅(B−u)≥0[pt⋅u+ht(w)+gt−1(w,atu)],gt(z,b)=Eft(z,b+dt),f_t(z,B) = \inf_{0\le u\le B,\ w = z-\mathbf 1\cdot(B-u)\ge 0}\bigl[p_t\cdot u + h_t(w) + g_{t-1}(w, a_t u)\bigr],\qquad g_t(z,b) = \mathbb E f_t(z, b+d_t),ft​(z,B)=0≤u≤B, w=z−1⋅(B−u)≥0inf​[pt​⋅u+ht​(w)+gt−1​(w,at​u)],gt​(z,b)=Eft​(z,b+dt​),

with g0(z,b)=v1(z)+v2(z−1⋅b)g_0(z,b) = v_1(z) + v_2(z-\mathbf 1\cdot b)g0​(z,b)=v1​(z)+v2​(z−1⋅b).

With δj\delta_jδj​ the jjj-th unit vector, the critical rationing level zˉtj∈[0,∞]\bar z_t^j\in[0,\infty]zˉtj​∈[0,∞] is +∞+\infty+∞ if φtj(w)=ptjw+ht(w)+gt−1(w,atwδj)\varphi_t^j(w) = p_t^j w + h_t(w) + g_{t-1}(w, a_t w\delta_j)φtj​(w)=ptj​w+ht​(w)+gt−1​(w,at​wδj​) is strictly decreasing on [0,∞)[0,\infty)[0,∞), and the smallest minimizer of φtj\varphi_t^jφtj​ on [0,∞)[0,\infty)[0,∞) otherwise. Under the paper's Theorem 1 the rationing level policy uj=(B(j)−z+zˉtj)+∧Bju^j = (B^{(j)} - z + \bar z_t^j)^+\wedge B^juj=(B(j)−z+zˉtj​)+∧Bj, with B(j)=∑i≥jBiB^{(j)} = \sum_{i\ge j}B^iB(j)=∑i≥j​Bi, is optimal: class jjj is served from stock only while the stock stays at or above zˉtj\bar z_t^jzˉtj​.

Formalization targets

Goal: Theorem 2 (p. 170)

Let 1≤t1\le t1≤t, t+1≤kt+1\le kt+1≤k, at+1=at=0a_{t+1} = a_t = 0at+1​=at​=0, and ai∈{0,1}a_i\in\{0,1\}ai​∈{0,1} for i≤ti\le ti≤t. If

pt+1j+lim⁡z→∞D+ht+1(z)≤ptj,thenzˉtj≤zˉt+1j.p_{t+1}^j + \lim_{z\to\infty}D^+h_{t+1}(z) \le p_t^j, \qquad\text{then}\qquad \bar z_t^j \le \bar z_{t+1}^j .pt+1j​+z→∞lim​D+ht+1​(z)≤ptj​,thenzˉtj​≤zˉt+1j​.

Milestones

  1. Lemma 5 (p. 168). For b≤bˉb\le\bar bb≤bˉ: Dz+gt(z,bˉ)≤Dz+gt(z,b)D_z^+g_t(z,\bar b)\le D_z^+g_t(z,b)Dz+​gt​(z,bˉ)≤Dz+​gt​(z,b).
  2. Lemma 6 (p. 168). For z≤zˉz\le\bar zz≤zˉ and y≥0y\ge 0y≥0: gt(zˉ,b)−gt(z,b)≤gt(zˉ+1⋅y,b+y)−gt(z+1⋅y,b+y)g_t(\bar z,b)-g_t(z,b)\le g_t(\bar z+\mathbf 1\cdot y,b+y)-g_t(z+\mathbf 1\cdot y,b+y)gt​(zˉ,b)−gt​(z,b)≤gt​(zˉ+1⋅y,b+y)−gt​(z+1⋅y,b+y).
  3. (12) (p. 170). If b=0b=0b=0 or at=1a_t=1at​=1, for small ε>0\varepsilon>0ε>0 and every d≥0d\ge0d≥0, the difference quotients of ft(⋅,b+d)f_t(\cdot,b+d)ft​(⋅,b+d), ft(⋅,b)f_t(\cdot,b)ft​(⋅,b) and ht+gt−1(⋅,b)h_t+g_{t-1}(\cdot,b)ht​+gt−1​(⋅,b) at zzz are ordered.
  4. Lemma 7 (p. 169). If at=1a_t = 1at​=1 or b=0b = 0b=0: Dz+gt(z,b)≤D+ht(z)+Dz+gt−1(z,b)D_z^+g_t(z,b)\le D^+h_t(z)+D_z^+g_{t-1}(z,b)Dz+​gt​(z,b)≤D+ht​(z)+Dz+​gt−1​(z,b).

Significance

The result says that, in intervals without backlogging, the stock reserved against a class is never larger late in the period than early in it, provided the class's penalty in the earlier interval plus the eventual marginal holding cost does not exceed its penalty in the later interval. Together with Theorem 1 it reduces the dynamic rationing problem to a family of one-dimensional critical numbers with a known order in both the class and the time index. The paper's example on p. 170 shows that the analogous monotonicity fails with backlogging, so the hypothesis at+1=at=0a_{t+1} = a_t = 0at+1​=at​=0 is essential.

The paper's proofs are complete but informal: they interchange expectation and right-derivative limits, and rest on Lemma 2 (convexity and attainment) and Theorem 1 (c). No part of this paper has been machine-checked. This mission produces formal statements of Theorem 2 and the three lemmas it rests on; contributions that formalize the known proofs, or that settle whether the extra hypothesis on earlier backlogging fractions (below) can be dropped, are equally in scope.

Difficulty

The central difficulty is Lemma 7, the comparison of the marginal value of stock in two consecutive intervals. The value functions are defined through an infimum and an expectation, so they are convex but not differentiable, and the minimizer in (1) changes regime each time the stock crosses a critical level; differentiating (1) directly is not available. Right derivatives may also be −∞-\infty−∞ at z=0z = 0z=0, and statements about them have to survive an interchange of expectation and a one-sided limit. The lemma depends on the optimal policy of Theorem 1, which is the subject of the first mission of this series. The claim of Lemma 7 is false when at=0a_t = 0at​=0 and b≠0b\ne0b=0 (p. 168), so no argument can ignore the case split.

Formalization scope

  • Classes are Fin n (paper class jjj is index j−1j-1j−1); intervals are natural numbers counted backwards; vectors are Fin n → ℝ with the pointwise order.
  • ftf_tft​ and gtg_tgt​ are defined by the recursion (1) with sInf and the Bochner integral, and every statement restricts to z≥0z\ge0z≥0, b≥0b\ge0b≥0, where the paper's Lemma 2 (proved in the first mission of the series) makes the infimum finite and attained and the integrand integrable.
  • D+D^+D+ is an extended-real liminf of right difference quotients, never a real derivWithin, which would return 000 where the right derivative is −∞-\infty−∞. Sums of right derivatives are taken in EReal.
  • The standing assumptions (A), (B), (C), at≥0a_t\ge0at​≥0, and probability laws on [0,∞)n[0,\infty)^n[0,∞)n with finite means are bundled in Model.Standing; (B)'s limit condition is read as "D+v2D^+v_2D+v2​ is bounded below".
  • Critical levels are values in WithTop ℝ satisfying the defining predicate (strictly decreasing ⇒ +∞+\infty+∞, otherwise the smallest minimizer); the theorem holds for any such choice. Their existence is a milestone of the first mission; a sanity check exhibits a model in which all hypotheses of Theorem 2 hold with zˉtj=zˉt+1j=0\bar z_t^j = \bar z_{t+1}^j = 0zˉtj​=zˉt+1j​=0.
  • The penalty hypothesis is stated for all z≥0z\ge0z≥0 instead of as a limit; these agree because D+ht+1D^+h_{t+1}D+ht+1​ is nondecreasing.
  • Disclosed addition: Theorem 2 is stated with ai∈{0,1}a_i\in\{0,1\}ai​∈{0,1} for all i≤ti\le ti≤t, the hypothesis of Lemma 7 that its proof uses; the printed statement names only at+1=at=0a_{t+1}=a_t=0at+1​=at​=0. The hypothesis on aia_iai​ in Lemmas 5–7 is required only for i≤ti\le ti≤t.
  • A formalization in which the goal assumes Lemma 7's inequality, or any property of D+gtD^+g_tD+gt​, would be trivial and is excluded: those facts appear only as milestones.

Selected references

  • D. M. Topkis, Optimal ordering and rationing policies in a nonstationary dynamic inventory model with n demand classes, Management Science 15(3) (1968), 160–176. https://doi.org/10.1287/mnsc.15.3.160
  • A. F. Veinott, Jr., Optimal policy in a dynamic, single product, nonstationary inventory model with several demand classes, Operations Research 13(5) (1965), 761–778. https://doi.org/10.1287/opre.13.5.761
7 thms1 active userReviewed
AnalysisOptimizationStochastic Systems·Captain: mikedeng1

Dynamic Scheduling with Convex Delay Costs: The Generalized cμ Rule: Policies Equalizing the Indices μ_k c_k(W_k/ρ_k) Attain the Heavy-Traffic Lower Bound on Cumulative Delay CostResearch Paper

Motivation

A single server shared by several classes of jobs must decide, at every moment, which class to work on. When each class kkk incurs a linear delay cost ckc_kck​ per unit of waiting time and needs mean service time 1/μk1/\mu_k1/μk​, the classical cμc\mucμ rule (serve the waiting class with the largest ckμkc_k\mu_kck​μk​) minimizes expected cost in many settings, from the M/G/1 queue to discrete-time models (Buyukkoc, Varaiya and Walrand, Adv. Appl. Probab. 1985). Linear costs are a strong restriction: in telecommunications, manufacturing and service systems the penalty for delay often grows faster than linearly, and with convex costs the static priority order of the cμc\mucμ rule is no longer optimal.

Van Mieghem (Ann. Appl. Probab. 1995) proposed the generalized cμc\mucμ rule: with nondecreasing convex delay costs CkC_kCk​ and marginal costs ck=Ck′c_k = C_k'ck​=Ck′​, serve the class with the largest index μkck(⋅)\mu_k c_k(\cdot)μk​ck​(⋅) evaluated at its current delay or workload. The paper proves that, in heavy traffic, this dynamic index policy is asymptotically optimal among all scheduling policies, including policies whose scaled workloads do not converge. This mission formalizes that result and the chain of propositions it rests on.

Setting

There are ddd job classes. In the nnnth system, the iiith class-kkk job arrives after an interarrival time uk,in>0u^n_{k,i}>0uk,in​>0 and needs service vk,in≥0v^n_{k,i}\ge 0vk,in​≥0. Akn(t)A^n_k(t)Akn​(t) counts class-kkk arrivals in [0,t][0,t][0,t] and Skn(x)S^n_k(x)Skn​(x) counts class-kkk completions within xxx units of server time. A policy is an allocation Tn=(Tkn)T^n=(T^n_k)Tn=(Tkn​), where Tkn(t)T^n_k(t)Tkn​(t) is the time in [0,t][0,t][0,t] spent on class kkk. From it come the headcount Nkn=Akn−Skn∘TknN^n_k = A^n_k - S^n_k\circ T^n_kNkn​=Akn​−Skn​∘Tkn​, the workload Wkn(t)=Vkn(Akn(t))−Tkn(t)W^n_k(t) = V^n_k(A^n_k(t)) - T^n_k(t)Wkn​(t)=Vkn​(Akn​(t))−Tkn​(t) (work present in class kkk), the class-FIFO delay τkn(t)=inf⁡{s≥0:Wkn(t)≤Tkn(t+s)−Tkn(t)}\tau^n_k(t)=\inf\{s\ge0: W^n_k(t)\le T^n_k(t+s)-T^n_k(t)\}τkn​(t)=inf{s≥0:Wkn​(t)≤Tkn​(t+s)−Tkn​(t)} of a job arriving at ttt, and the cumulative cost Jn(t)=∑k∑i≤Akn(t)Ckn(τk,in)J^n(t)=\sum_k\sum_{i\le A^n_k(t)} C^n_k(\tau^n_{k,i})Jn(t)=∑k​∑i≤Akn​(t)​Ckn​(τk,in​). Feasible policies are continuous, nondecreasing and work conserving.

The nnnth system runs on [0,n][0,n][0,n] and is studied at the diffusion scale: W~kn(t)=Wkn(nt)/n\tilde W^n_k(t)=W^n_k(nt)/\sqrt nW~kn​(t)=Wkn​(nt)/n​, N~kn\tilde N^n_kN~kn​, τ~kn\tilde\tau^n_kτ~kn​ likewise, and J~n(t)=Jn(nt)/n\tilde J^n(t)=J^n(nt)/nJ~n(t)=Jn(nt)/n. Arrival and service processes expand as An(nt)=nAˉn(t)+n A~n(t)+o(n)A^n(nt)=n\bar A^n(t)+\sqrt n\,\tilde A^n(t)+o(\sqrt n)An(nt)=nAˉn(t)+n​A~n(t)+o(n​) and Sn(nt)=nSˉn(t)+n S~n(t)+o(n)S^n(nt)=n\bar S^n(t)+\sqrt n\,\tilde S^n(t)+o(\sqrt n)Sn(nt)=nSˉn(t)+n​S~n(t)+o(n​). Assumption 1 asks that these terms converge, with rates λ=Aˉ∗′>0\lambda=\bar A^{*\prime}>0λ=Aˉ∗′>0, μ=Sˉ∗′>0\mu=\bar S^{*\prime}>0μ=Sˉ∗′>0, and that the system is in heavy traffic: n(R+n−e)→c~∗\sqrt n(R^n_+-e)\to\tilde c^*n​(R+n​−e)→c~∗, where Rkn=(Sˉkn)−1∘AˉknR^n_k=(\bar S^n_k)^{-1}\circ\bar A^n_kRkn​=(Sˉkn​)−1∘Aˉkn​ and ρk=λk/μk\rho_k=\lambda_k/\mu_kρk​=λk​/μk​ is the traffic intensity. Assumption 2 asks that the costs scale, Cn(n ⋅)→C∗(⋅)C^n(\sqrt n\,\cdot)\to C^*(\cdot)Cn(n​⋅)→C∗(⋅), and Assumption 3 that C∗C^*C∗ be strictly convex and C1\mathcal C^1C1.

The scaled total workload has a policy-independent limit W~+∗=φ(L~+∗+c~∗)\tilde W^*_+=\varphi(\tilde L^*_++\tilde c^*)W~+∗​=φ(L~+∗​+c~∗), with φ\varphiφ the one-dimensional reflection map. The key static problem splits a total yyy among classes:

g∘y(t)=arg⁡min⁡x∈R+d, ∑kxk=y(t) ∑kλk(t) Ck∗(xkρk(t)).(43)g\circ y(t)=\arg\min_{x\in\mathbb R^d_+,\ \sum_k x_k=y(t)}\ \sum_k\lambda_k(t)\,C^*_k\Big(\frac{x_k}{\rho_k(t)}\Big). \tag{43}g∘y(t)=argx∈R+d​, ∑k​xk​=y(t)min​ k∑​λk​(t)Ck∗​(ρk​(t)xk​​).(43)

Formalization targets

Goal: Proposition 7

Under Assumptions 1–3, for feasible policies with

max⁡k,l∥μkck∗(W~knρk)−μlcl∗(W~lnρl)∥→0,(51)\max_{k,l}\Big\|\mu_kc^*_k\Big(\frac{\tilde W^n_k}{\rho_k}\Big)-\mu_lc^*_l\Big(\frac{\tilde W^n_l}{\rho_l}\Big)\Big\|\to0, \tag{51}k,lmax​​μk​ck∗​(ρk​W~kn​​)−μl​cl∗​(ρl​W~ln​​)​→0,(51)

the costs and delays converge,

J~n→J~∗=∑k∫λkCk∗([g∘W~+∗]kρk)dt,τ~n→g∘W~+∗ρ.(52–53)\tilde J^n\to\tilde J^*=\sum_k\int\lambda_kC^*_k\Big(\frac{[g\circ\tilde W^*_+]_k}{\rho_k}\Big)dt,\qquad \tilde\tau^n\to\frac{g\circ\tilde W^*_+}{\rho}. \tag{52–53}J~n→J~∗=k∑​∫λk​Ck∗​(ρk​[g∘W~+∗​]k​​)dt,τ~n→ρg∘W~+∗​​.(52–53)

Milestones

Lemma 1 and Proposition 2 (jumps (33)–(34), first-order allocation (27), total workload (32), boundedness, equivalences (35)); Proposition 3 (μkW~kn−N~kn→0\mu_k\tilde W^n_k-\tilde N^n_k\to0μk​W~kn​−N~kn​→0); Proposition 4 (Little's law); Proposition 5 (converging policies); uniqueness and continuity of ggg (§4.1); the Kuhn–Tucker characterization (46)–(50); and Proposition 6, the lower bound

lim inf⁡nJ~n(t)≥J~∗(t)for every feasible policy sequence.(44)\liminf_n\tilde J^n(t)\ge\tilde J^*(t)\qquad\text{for every feasible policy sequence.}\tag{44}nliminf​J~n(t)≥J~∗(t)for every feasible policy sequence.(44)

Proposition 6 is what makes Proposition 7 an optimality statement; it is a separate milestone and not part of the goal.

Significance

The result. Proposition 7 identifies a simple, myopic index rule that is optimal at the diffusion scale for every time simultaneously, for arbitrary convex delay costs and nonstationary rates. It extends the cμc\mucμ rule beyond linear costs, links it to the "hug the optimal workload curve" policies of heavy-traffic control, and shows that the instantaneous allocation problem (43) is all that matters asymptotically. Later work on Brownian control of multiclass queues and on convex-cost scheduling (Mandelbaum and Stolyar 2004) builds on this.

Formalizing it. The paper's proofs are short and informal at several points, and the formalization makes these points explicit: the domain of every uniform limit, the policy class, and the reading of μ\muμ. No part of this paper has a machine-checked proof. A completed mission would give a verified heavy-traffic analysis of a multiclass single-server queue at the sample-path level, the first on the platform with a deterministic diffusion scaling and an explicit reflection map.

Difficulty

The obvious route is to show that W~n\tilde W^nW~n converges and then pass to the limit in the cost. That fails for the lower bound: Proposition 6 must cover policies whose class workloads never converge (polling-type policies), so it cannot use compactness of W~n\tilde W^nW~n and must instead compare the cost of an arbitrary split of W~+n\tilde W^n_+W~+n​ with the optimal split, locally in time. For the goal, (51) controls only the indices, so the convergence of W~n\tilde W^nW~n must be extracted from (51) through the inverse marginal costs, the convergence of the total workload, and the Kuhn–Tucker conditions of (43). Delay convergence then needs a uniform control of T~n\tilde T^nT~n over windows of length O(n−1/2)O(n^{-1/2})O(n−1/2), and cost convergence needs the convergence of the costs CnC^nCn applied to delays of order n\sqrt nn​.

Formalization scope

Lean namespace GenCMu.HeavyTraffic. The development is deterministic and works one sample path at a time; the stochastic version (Proposition 8) is not part of this mission. Conventions committed to:

  • Classes are Fin d; job indices start at 111; every "→" between functions is uniform convergence ((11)), on [0,1][0,1][0,1] unless stated; lim inf⁡\liminfliminf is written as "eventually ≥\ge≥ bound −ε-\varepsilon−ε".
  • The scaled processes are defined exactly, with Tˉn=Lˉn=Rn\bar T^n=\bar L^n=R^nTˉn=Lˉn=Rn ((69), (75)); the inverse in (14) is the generalized inverse on [0,1][0,1][0,1].
  • The o(n1/2)o(n^{1/2})o(n1/2) terms of (12)–(13) are uniform in ttt. The trends converge in C1\mathcal C^1C1, not only in C\mathcal CC. Without this, Aˉn,Sˉn\bar A^n,\bar S^nAˉn,Sˉn can oscillate on the n−1/2n^{-1/2}n−1/2 scale and (33)–(34) and Proposition 3 fail. The service expansion holds on server time [0,2n][0,2n][0,2n], so that for d=1d=1d=1 in heavy traffic the service times of jobs arriving before nnn are covered.
  • μk\mu_kμk​ in (20), (36), (41), (46)–(51) is μk(Rk∗(t))\mu_k(R^*_k(t))μk​(Rk∗​(t)), the service rate in real time, so ρk=λk/μk(Rk∗)\rho_k=\lambda_k/\mu_k(R^*_k)ρk​=λk​/μk​(Rk∗​).
  • (38) is uniform on compacts. Assumption 3's interior clause is imposed where W~+∗(t)>0\tilde W^*_+(t)>0W~+∗​(t)>0. At W~+∗(t)=0\tilde W^*_+(t)=0W~+∗​(t)=0 no point of Ω={0}\Omega=\{0\}Ω={0} is interior, and the literal clause would make the goal vacuous.
  • For the heavy-traffic limit and goal, feasible policies satisfy F1 (without adaptedness), F2–F4, Wk≥0W_k\ge0Wk​≥0, work conservation (used by W+n=φ(Xn)W^n_+=\varphi(X^n)W+n​=φ(Xn)), and every job finishes. Proposition 6 uses a broader cost-admissible class that permits idleness with work waiting. Class-FIFO is built into the delay (9).
  • The delays of jobs arriving near the horizon depend on post-horizon service. We require the allocation on every bounded diffusion-scale window after nnn to advance at the first-order rates ρk(1)\rho_k(1)ρk​(1). This explicit boundary hypothesis keeps Propositions 2, 4, 5, and 7 on the paper's full [0,1][0,1][0,1].
  • Lemma 1 additionally assumes a common Lipschitz bound on the first-order partial-sum trends. Uniform convergence alone permits jumps of order n\sqrt nn​; the bound repairs that omission. Uniqueness of the solution of (43) is stated under strict convexity: "convex increasing" (p. 820) does not give it. (35) is stated without the implication from τ~n\tilde\tau^nτ~n back to W~n\tilde W^nW~n, which fails in the uniform topology.
  • J~∗\tilde J^*J~∗ integrates the optimal value of (43), so Proposition 6 needs no choice of minimizer.

A trivializing formalization is ruled out: no theorem assumes the convergence of W~n\tilde W^nW~n or W~+n\tilde W^n_+W~+n​, the uniqueness of minimizers, or the lower bound, and the joint satisfiability of all hypotheses has been checked in Lean for d=1d=1d=1.

Needed infrastructure: sample-path reflection-map estimates, counting/partial-sum inversion with uniform o(n)o(\sqrt n)o(n​) errors, parametric convex optimization (Berge's theorem for (43)), and Riemann-sum arguments for the lower bound. The reflection-map and inversion lemmas are reusable beyond this mission. Contributions to any milestone are welcome. The definitions are frozen, so a proof that needs a different definition should be raised in the discussion.

Selected references

  • J. A. Van Mieghem, Dynamic Scheduling with Convex Delay Costs: The Generalized cμ Rule, Ann. Appl. Probab. 5(3) (1995) 809–833. https://doi.org/10.1214/aoap/1177004706
  • C. Buyukkoc, P. Varaiya, J. Walrand, The cμ rule revisited, Adv. Appl. Probab. 17 (1985) 237–238.
  • J. M. Harrison, Brownian Motion and Stochastic Flow Systems, Wiley, 1985 (reflection map).
  • D. L. Iglehart, W. Whitt, Multiple channel queues in heavy traffic. I, Adv. Appl. Probab. 2 (1970) 150–177.
  • A. Mandelbaum, A. L. Stolyar, Scheduling flexible servers with convex delay costs: heavy-traffic optimality of the generalized cμ-rule, Oper. Res. 52(6) (2004) 836–855. https://doi.org/10.1287/opre.1040.0156
16 thms1 active userReviewed
CombinatoricsProbability·Captain: mikedeng1

Concentration of Measure and Isoperimetric Inequalities in Product Spaces XI: The Random Assignment Problem — Fluctuations of the Optimal Cost of Order (log N)²/(√N log log N)Textbook

Motivation

The assignment problem asks for a one-to-one matching of NNN agents to NNN tasks that minimizes the total cost. It is one of the basic problems of combinatorial optimization and is solved in polynomial time by the Hungarian method. Its random version, in which the N2N^2N2 costs are independent random variables, is a standard model for the typical behavior of optimization problems on random inputs, and it has been studied in probability, operations research and statistical physics.

Timeline:

  • Walkup (1979) showed that the expected optimal cost E(LN)E(L_N)E(LN​) of the random assignment problem with independent uniform [0,1][0,1][0,1] costs stays bounded as N→∞N \to \inftyN→∞ (doi:10.1137/0208036).
  • Karp (1987) proved E(LN)≤2E(L_N) \le 2E(LN​)≤2.
  • Talagrand (1995), in the article this mission formalizes, bounded the fluctuations of LNL_NLN​ around its median by K(log⁡N)2/(Nlog⁡log⁡N)K(\log N)^2/(\sqrt N \log\log N)K(logN)2/(N​loglogN), as a showcase of his concentration inequality for product measures (doi:10.1007/BF02699376).
  • Aldous (2001) proved the limit E(LN)→π2/6E(L_N) \to \pi^2/6E(LN​)→π2/6, the value predicted by Mézard and Parisi (doi:10.1002/rsa.1015).

This mission is Chapter 10 of Talagrand's article.

Setting

Fix NNN and two sets III and JJJ of cardinal NNN. An assignment is a one-to-one map τ\tauτ from III onto JJJ. Given a cost matrix a=(ai,j)i∈I,j∈Ja = (a_{i,j})_{i \in I, j \in J}a=(ai,j​)i∈I,j∈J​, the cost of τ\tauτ is ∑i∈Iai,τ(i)\sum_{i \in I} a_{i,\tau(i)}∑i∈I​ai,τ(i)​. In the random problem the costs are Xi,jX_{i,j}Xi,j​, independent and uniformly distributed on [0,1][0,1][0,1], and the optimal cost is the random variable

LN=min⁡{∑i∈IXi,τ(i)  ;  τ assignment}.L_N = \min\Big\{ \sum_{i \in I} X_{i,\tau(i)} \;;\; \tau \text{ assignment} \Big\}.LN​=min{i∈I∑​Xi,τ(i)​;τ assignment}.

A median of LNL_NLN​ is any real MMM with P(LN≤M)≥1/2P(L_N \le M) \ge 1/2P(LN​≤M)≥1/2 and P(LN≥M)≥1/2P(L_N \ge M) \ge 1/2P(LN​≥M)≥1/2.

The argument uses digraphs. A digraph is a subset D⊆I×JD \subseteq I \times JD⊆I×J, and for S⊆IS \subseteq IS⊆I the set of neighbors is D(S)={j∈J; ∃i∈S, (i,j)∈D}D(S) = \{ j \in J ;\ \exists i \in S,\ (i,j) \in D \}D(S)={j∈J; ∃i∈S, (i,j)∈D}. For α≥2\alpha \ge 2α≥2, DDD is α\alphaα-expanding if for every S⊆IS \subseteq IS⊆I

card⁡S≤N2⇒card⁡D(S)≥min⁡(αcard⁡S,N2),card⁡S≥N2⇒card⁡D(S)≥N−1α(N−card⁡S).\operatorname{card} S \le \tfrac N2 \Rightarrow \operatorname{card} D(S) \ge \min\big(\alpha \operatorname{card} S, \tfrac N2\big), \qquad \operatorname{card} S \ge \tfrac N2 \Rightarrow \operatorname{card} D(S) \ge N - \tfrac 1\alpha (N - \operatorname{card} S).cardS≤2N​⇒cardD(S)≥min(αcardS,2N​),cardS≥2N​⇒cardD(S)≥N−α1​(N−cardS).

For u>0u > 0u>0 the threshold digraph is Du={(i,j); Xi,j≤2uN−1log⁡N}D_u = \{(i,j) ;\ X_{i,j} \le 2uN^{-1}\log N\}Du​={(i,j); Xi,j​≤2uN−1logN}: it keeps only the cheap edges.

Formalization targets

Goal: Theorem 10.5

There is a universal constant KKK such that for N≥3N \ge 3N≥3, every median MMM of LNL_NLN​ and every t≥0t \ge 0t≥0,

t≤log⁡N⇒P(∣LN−M∣≥Kt(log⁡N)2Nlog⁡log⁡N)≤2e−t2,t \le \sqrt{\log N} \Rightarrow P\Big(|L_N - M| \ge \frac{K t (\log N)^2}{\sqrt N \log\log N}\Big) \le 2 e^{-t^2},t≤logN​⇒P(∣LN​−M∣≥N​loglogNKt(logN)2​)≤2e−t2, t≥log⁡N⇒P(∣LN−M∣≥Kt3log⁡NNlog⁡t2)≤2e−t2.t \ge \sqrt{\log N} \Rightarrow P\Big(|L_N - M| \ge \frac{K t^3 \log N}{\sqrt N \log t^2}\Big) \le 2 e^{-t^2}.t≥logN​⇒P(∣LN​−M∣≥N​logt2Kt3logN​)≤2e−t2.

The constant KKK is left unspecified, as in the paper.

Milestones, in attack order

  1. Lemma 10.1 (deterministic): in an α\alphaα-expanding digraph with αm≥N/2\alpha^m \ge N/2αm≥N/2, every i∈Ii \in Ii∈I lies on a cycle i=i1,…,in+1=ii = i_1, \dots, i_{n+1} = ii=i1​,…,in+1​=i of distinct points, n≤2mn \le 2mn≤2m, with (iℓ,τ(iℓ+1))∈D(i_\ell, \tau(i_{\ell+1})) \in D(iℓ​,τ(iℓ+1​))∈D.
  2. Corollary 10.2 (deterministic in the costs): if DuD_uDu​ is α\alphaα-expanding and αm≥N/2\alpha^m \ge N/2αm≥N/2, every optimal assignment satisfies Xi,τ(i)≤4muN−1log⁡NX_{i,\tau(i)} \le 4muN^{-1}\log NXi,τ(i)​≤4muN−1logN.
  3. Lemma 10.4: for independent events of probability ppp, fewer than δpN\delta p NδpN occur with probability at most exp⁡(−Np/K(δ))\exp(-Np/K(\delta))exp(−Np/K(δ)).
  4. Proposition 10.3: for a universal KKK and u>Ku > Ku>K with ulog⁡N≤Nu \log N \le NulogN≤N, DuD_uDu​ is ulog⁡Nu\log NulogN-expanding with probability at least 1−N−u/K1 - N^{-u/K}1−N−u/K.

Significance

The expected value of LNL_NLN​ is of the same order as a single cost, yet LNL_NLN​ is a function of N2N^2N2 independent variables. A generic bounded-difference bound in all N2N^2N2 coordinates gives no useful concentration at this scale. Theorem 10.5 shows that the fluctuations are at most of order (log⁡N)2/(Nlog⁡log⁡N)(\log N)^2/(\sqrt N \log\log N)(logN)2/(N​loglogN), and by the remark after the theorem, so is the standard deviation of LNL_NLN​. The chapter is a worked example of how a concentration inequality in product spaces is combined with a combinatorial localization argument, the reduction to small costs, and that pattern recurs in the study of random combinatorial optimization problems.

Status: the theorem and its lemmas are proved in the paper. As far as a search of the platform shows, none of them has a machine-checked proof. The two posed statements of Theorem 4.1.1 on the platform (finite-alphabet form) are the concentration inequality the proof calls on; formalizing this chapter adds the combinatorics of expanding digraphs, the random-graph estimate Proposition 10.3 and the final assembly.

Difficulty

The naive approach applies a concentration inequality directly to LNL_NLN​ as a function of the N2N^2N2 costs, each of which can change LNL_NLN​ by up to 111. That loses all control, because the variance proxy grows with the number of coordinates. The central step is to show that, with high probability, an optimal assignment only uses costs of order N−1(log⁡N)2N^{-1}(\log N)^2N−1(logN)2. This requires the cycle structure of Lemma 10.1, and it requires showing that the sparse random digraph DuD_uDu​ has expansion uniformly over all subsets SSS of III, whose number is exponential in NNN. Proposition 10.3 has to beat that union bound, and its proof covers only one of the two expansion conditions in detail; the condition (10.2) for large sets is "left to the reader". The final step combines the truncation with Theorem 4.1.1 and a careful choice of parameters for the two ranges of ttt.

Formalization scope

The Lean development lives in the namespace TalagrandConc.Assignment and commits to the following conventions:

  • I=J=I = J = I=J= Fin N, assignments are Equiv.Perm (Fin N), and LNL_NLN​ is optCost, a Finset.inf' over all permutations (no junk value).
  • The probability space is the cost matrices Fin N × Fin N → ℝ with costLaw N, the product of Lebesgue measure restricted to [0,1][0,1][0,1]. The coordinates are the Xi,jX_{i,j}Xi,j​.
  • Cardinalities of subsets are Set.ncard on the finite type Fin N. α\alphaα-expansion includes α≥2\alpha \ge 2α≥2 as part of the definition.
  • Probabilities are in ℝ≥0∞, and 1−N−u/K1 - N^{-u/K}1−N−u/K is truncated subtraction there.
  • Universal constants are existentially quantified before every object they are uniform over. K(δ)K(\delta)K(δ) in Lemma 10.4 is chosen after δ\deltaδ and before everything else.
  • Implicit side conditions of the paper are stated: m≥1m \ge 1m≥1 in Lemma 10.1 and Corollary 10.2, n≥1n \ge 1n≥1 for the cycle, N≥1N \ge 1N≥1 in Proposition 10.3, t≥0t \ge 0t≥0 in Theorem 10.5.

A cycle of length n=0n = 0n=0 would satisfy Lemma 10.1 vacuously. The formal statement therefore requires n≥1n \ge 1n≥1, which rules out that trivializing reading. Similarly, Theorem 10.5 fixes one constant KKK for all NNN; a KKK depending on NNN would make the bound empty.

Infrastructure: the product of uniform laws on a finite index set, iterated expansion arguments for bipartite digraphs, a lower-tail Chernoff bound for independent events, and the union bound over subsets are reusable beyond this mission. Theorem 4.1.1 of the paper is posed elsewhere on the platform and is not restated here. The bound (8.1.1) used in Step 2 of the proof belongs to another chapter of the series. Contributions of proofs of any milestone, or of intermediate displayed estimates such as (10.5), (10.6) and (10.9), are welcome.

Selected references

  • M. Talagrand, Concentration of measure and isoperimetric inequalities in product spaces, Publ. Math. IHÉS 81 (1995), 73–205. doi:10.1007/BF02699376
  • D. W. Walkup, On the expected value of a random assignment problem, SIAM J. Comput. 8 (1979), 440–442. doi:10.1137/0208036
  • R. M. Karp, An upper bound on the expected cost of an optimal assignment, in Discrete Algorithms and Complexity, Academic Press (1987), 1–4.
  • J. M. Steele, Probability Theory and Combinatorial Optimization, SIAM (1997). doi:10.1137/1.9781611970029
  • D. J. Aldous, The ζ(2) limit in the random assignment problem, Random Structures Algorithms 18 (2001), 381–418. doi:10.1002/rsa.1015
7 thms1 active userReviewed
Algorithmic Game TheoryMechanism Design·Captain: mikedeng1

Incentive Compatibility and the Bargaining Problem II: Every Equilibrium of Every Choice Mechanism Yields an Incentive-Feasible Allocation, F** = F*Research Paper

Motivation

An arbitrator who must choose among collective options for a group of players usually does not know the players' private characteristics. The arbitrator can only ask, and a player who expects to profit from a false answer may give one. Before designing any procedure, the arbitrator therefore needs to know which expected-payoff allocations can be reached at all once strategic misreporting is taken into account.

Myerson (1979) answers this for Bayesian collective choice problems with finitely many types. Restricting attention to mechanisms that simply ask each player for their type, and that make honest answers optimal, loses nothing: every expected-payoff allocation that any mechanism can produce in equilibrium is already produced honestly by such a direct mechanism, and conversely. This is the revelation principle in the form used throughout Bayesian mechanism design. It is what makes incentive-compatibility constraints the starting point for auction design, bilateral trade, and the bargaining solution in the second half of the same paper.

Timeline. Gibbard (1973) gave the dominant-strategy form of the principle. Rosenthal (Review of Economic Studies, 1978) studied arbitration of two-party disputes under uncertainty; Myerson notes that Theorem 2 could be derived as a corollary of Rosenthal's Theorem 3. Dasgupta, Hammond and Maskin (1979) gave general Bayesian implementation results in the same year. Myerson's Theorem 2 states the principle as an equality of interim payoff sets over all finite response sets. Myerson (1982) later extended it to principal–agent problems with moral hazard.

Setting

A Bayesian collective choice problem (C,A1,…,An,U1,…,Un,P)(C,A_1,\dots,A_n,U_1,\dots,U_n,P)(C,A1​,…,An​,U1​,…,Un​,P) has a nonempty finite set of players iii, a nonempty finite set AiA_iAi​ of types for each player, a nonempty finite set CCC of collective choices, utilities Ui(c,α)U_i(c,\alpha)Ui​(c,α) depending on the choice and on the true type profile α=(α1,…,αn)\alpha=(\alpha_1,\dots,\alpha_n)α=(α1​,…,αn​), and a probability distribution PPP on type profiles. With Ri(ai)=∑β:βi=aiP(β)R_i(a_i)=\sum_{\beta:\beta_i=a_i}P(\beta)Ri​(ai​)=∑β:βi​=ai​​P(β) the marginal of type aia_iai​, a player of type aia_iai​ assigns the conditional probability Pi(α∣ai)=P(α)/Ri(ai)P_i(\alpha\mid a_i)=P(\alpha)/R_i(a_i)Pi​(α∣ai​)=P(α)/Ri​(ai​) to profiles with αi=ai\alpha_i=a_iαi​=ai​, and 000 to the others.

A choice mechanism on response sets S1,…,SnS_1,\dots,S_nS1​,…,Sn​ (nonempty finite sets of possible answers) is a function π(c∣s)≥0\pi(c\mid s)\ge0π(c∣s)≥0 with ∑cπ(c∣s)=1\sum_{c}\pi(c\mid s)=1∑c​π(c∣s)=1 for every response profile sss. On the standard response sets Si=AiS_i=A_iSi​=Ai​, the payoff of type aia_iai​ who reports bib_ibi​ while the others are honest is

Zi(π,bi∣ai)=∑α∑cPi(α∣ai) π(c∣α−i,bi) Ui(c,α).Z_i(\pi,b_i\mid a_i)=\sum_\alpha\sum_{c}P_i(\alpha\mid a_i)\,\pi(c\mid\alpha_{-i},b_i)\,U_i(c,\alpha).Zi​(π,bi​∣ai​)=α∑​c∑​Pi​(α∣ai​)π(c∣α−i​,bi​)Ui​(c,α).

The mechanism is Bayesian incentive-compatible if Zi(π,ai∣ai)≥Zi(π,bi∣ai)Z_i(\pi,a_i\mid a_i)\ge Z_i(\pi,b_i\mid a_i)Zi​(π,ai​∣ai​)≥Zi​(π,bi​∣ai​) for all i,ai,bii,a_i,b_ii,ai​,bi​. Its honest allocation is the vector V(π)V(\pi)V(π) with coordinates Vi(π∣ai)=Zi(π,ai∣ai)V_i(\pi\mid a_i)=Z_i(\pi,a_i\mid a_i)Vi​(π∣ai​)=Zi​(π,ai​∣ai​), indexed by the disjoint union of the AiA_iAi​. The incentive-feasible set is

F∗={V(π):π a Bayesian incentive-compatible choice mechanism on the standard response sets}.F^*=\{V(\pi):\pi\text{ a Bayesian incentive-compatible choice mechanism on the standard response sets}\}.F∗={V(π):π a Bayesian incentive-compatible choice mechanism on the standard response sets}.

On general response sets, a response plan σi(si∣ai)\sigma_i(s_i\mid a_i)σi​(si​∣ai​) is a probability distribution over SiS_iSi​ for each type aia_iai​. Plans σ=(σ1,…,σn)\sigma=(\sigma_1,\dots,\sigma_n)σ=(σ1​,…,σn​) give type aia_iai​ the payoff

Wi(π,σ∣ai)=∑α∑s∑cPi(α∣ai)(∏jσj(sj∣αj))π(c∣s) Ui(c,α).W_i(\pi,\sigma\mid a_i)=\sum_\alpha\sum_s\sum_c P_i(\alpha\mid a_i)\Big(\prod_{j}\sigma_j(s_j\mid\alpha_j)\Big)\pi(c\mid s)\,U_i(c,\alpha).Wi​(π,σ∣ai​)=α∑​s∑​c∑​Pi​(α∣ai​)(j∏​σj​(sj​∣αj​))π(c∣s)Ui​(c,α).

They form a response-plan equilibrium for π\piπ if no type aia_iai​ of any player gains by switching to any other response plan σi′\sigma_i'σi′​. The equilibrium-feasible set F∗∗F^{**}F∗∗ collects W(π,σ)W(\pi,\sigma)W(π,σ) over all choice mechanisms π\piπ, on all nonempty finite response sets, and all response-plan equilibria σ\sigmaσ for π\piπ.

Formalization targets

Goal: Theorem 2

F∗∗=F∗.F^{**}=F^*.F∗∗=F∗.

Both inclusions are asserted for every Bayesian collective choice problem. F∗∗⊆F∗F^{**}\subseteq F^*F∗∗⊆F∗ says equilibrium behaviour under any mechanism can be replicated honestly by an incentive-compatible direct mechanism. F∗⊆F∗∗F^*\subseteq F^{**}F∗⊆F∗∗ says honest reporting is itself an equilibrium of every incentive-compatible mechanism.

Milestones (the steps of the paper's proof)

  1. For any choice mechanism π\piπ and response plans σ\sigmaσ, the induced direct mechanism π′(c∣α)=∑sπ(c∣s)∏iσi(si∣αi)\pi'(c\mid\alpha)=\sum_s\pi(c\mid s)\prod_i\sigma_i(s_i\mid\alpha_i)π′(c∣α)=∑s​π(c∣s)∏i​σi​(si​∣αi​) is a choice mechanism and V(π′)=W(π,σ)V(\pi')=W(\pi,\sigma)V(π′)=W(π,σ).
  2. If σ\sigmaσ is a response-plan equilibrium for π\piπ, then π′\pi'π′ is Bayesian incentive-compatible.
  3. If π′\pi'π′ is a Bayesian incentive-compatible choice mechanism, the honest plans σi′(bi∣ai)=1[bi=ai]\sigma_i'(b_i\mid a_i)=\mathbf 1[b_i=a_i]σi′​(bi​∣ai​)=1[bi​=ai​] form a response-plan equilibrium for π′\pi'π′, and W(π′,σ′)=V(π′)W(\pi',\sigma')=V(\pi')W(π′,σ′)=V(π′).

Significance

The result. Theorem 2 reduces a search over all communication protocols, an unbounded family of response sets and mixed reporting behaviours, to a finite system of linear inequalities in π\piπ on a fixed finite domain. Combined with Theorem 1 of the paper, which shows F∗F^*F∗ is a nonempty compact convex set, it justifies applying a bargaining solution to F∗F^*F∗ rather than to some larger, ill-defined set of attainable outcomes. The same reduction underlies the use of incentive constraints in optimal auctions, bilateral trade and correlated-type mechanism design.

Formalizing it. The result is classical and its proof is short, but it is a statement about all finite response sets at once, and the equality of two sets of payoff vectors is sensitive to the conventions: which plans are allowed as deviations, where the conditioning happens, and whether the induced mechanism is a genuine probability distribution. A machine-checked version pins these down. To our knowledge no formal proof of this finite Bayesian revelation principle exists. The platform has related, unproved items in other models: MechanismDesign.Robust.revelation_principle (Börgers, Prop. 10.2, on a countable Harsanyi type space with PMF lotteries; one inclusion, outcome equivalence) and single-agent, quasi-linear and dominant-strategy revelation principles from the same book. None of them is the statement here.

Difficulty

The mathematics consists of finite sums, but the bookkeeping is the obstacle. Milestone 1 requires exchanging a sum over response profiles with a product over players and using that each plan sums to one. Milestone 2 requires recognizing a lie bib_ibi​ in π′\pi'π′ as a particular whole response plan in π\piπ: the plan that, at every type, behaves as type bib_ibi​ would. Because Pi(α∣ai)P_i(\alpha\mid a_i)Pi​(α∣ai​) vanishes off αi=ai\alpha_i=a_iαi​=ai​, only its value at aia_iai​ matters. For milestone 3, a mixed deviation from honesty has to be bounded by the best pure misreport, a convex-combination argument over AiA_iAi​. Identifying the F∗∗F^{**}F∗∗ side of the goal also needs instances on an existentially chosen family of types.

Formalization scope

All objects live in MyersonBargaining.Revelation. Players form a nonempty finite type ι; types A i, choices C and response sets S i are nonempty finite types in Type. Mechanisms and plans are real-valued functions with the probability constraints as predicates (IsChoiceMechanism, IsResponsePlan), event first and condition second: π c s, σ i s a. Allocation vectors are functions on Σ i, A i. The paper's loose phrases are read as follows:

  • "probability distribution" means nonnegativity and sum one;
  • (3) divides by Ri(ai)R_i(a_i)Ri​(ai​), so the problem requires Ri(ai)>0R_i(a_i)>0Ri​(ai​)>0; individual profiles may have probability zero;
  • the consistent (common-prior) reading of (3)–(4) is formalized, not the subjective reading the paper permits on p. 63;
  • "for every possible alternative response plan" in (14) ranges over all mixed response plans, compared type by type;
  • "π\piπ is a choice mechanism" in (15) ranges over all nonempty finite response sets, with an explicit existential over the family S : ι → Type and its instances;
  • "incentive-compatible mechanism" means a choice mechanism satisfying (6).

The product in (12) is printed with σj(sj∣aj)\sigma_j(s_j\mid a_j)σj​(sj​∣aj​); it is formalized with αj\alpha_jαj​, as the paper's own induced mechanism on p. 66 uses. Milestone 1 is stated for arbitrary response plans, not only for equilibria. Section 6's numerical example is not formalized.

A trivializing formalization is ruled out explicitly: fixing Si=AiS_i=A_iSi​=Ai​ in F∗∗F^{**}F∗∗, restricting deviations to pure plans, or stating only F∗∗⊆F∗F^{**}\subseteq F^*F∗∗⊆F∗ would each change the theorem, and none is done.

A complete development needs finite sums over dependent product types (Fintype.piFinset, Finset.prod_univ_sum), Function.update on dependent families, and nothing beyond Mathlib. The two definition modules (the model and the response-plan objects) are reusable by any finite Bayesian mechanism-design formalization. Proofs of any milestone, alternative proofs via Rosenthal's argument, and cleaner restatements as lemmas about stochastic matrices are welcome.

Selected references

  • Roger B. Myerson, Incentive compatibility and the bargaining problem, Econometrica 47(1), 61–73, 1979. https://doi.org/10.2307/1912346
  • Robert W. Rosenthal, Arbitration of two-party disputes under uncertainty, Review of Economic Studies 45(3), 1978 (cited in the source as reference [8], "forthcoming").
  • Allan Gibbard, Manipulation of voting schemes: a general result, Econometrica 41(4), 587–601, 1973. https://doi.org/10.2307/1914083
  • Partha Dasgupta, Peter Hammond, Eric Maskin, The implementation of social choice rules: some general results on incentive compatibility, Review of Economic Studies 46(2), 185–216, 1979. https://doi.org/10.2307/2297045
  • Roger B. Myerson, Optimal coordination mechanisms in generalized principal–agent problems, Journal of Mathematical Economics 10(1), 67–81, 1982. https://doi.org/10.1016/0304-4068(82)90006-4
6 thms1 active userReviewed
ProbabilityStochastic Systems·Captain: mikedeng1

Regenerative Stochastic Processes 2: The Ergodic Theorem — a Cumulative Process Satisfies w_t/t → κ₁/μ₁ with Probability OneResearch Paper

Motivation

Many quantities in operations research accumulate over time in a system that periodically starts afresh: the cost incurred by an inventory system that returns to the same stock level, the work completed by a server between successive moments it empties, the reward collected by a machine between repairs. The basic question about such a quantity is its long-run rate: does the amount accumulated by time ttt, divided by ttt, settle down, and to what?

W. L. Smith's 1955 paper Regenerative stochastic processes (DOI 10.1098/rspa.1955.0198) introduced the class of cumulative processes to answer this question, and §5·3 of the paper proves the answer, Theorem 7, which Smith calls "the ergodic theorem": the long-run rate equals the expected accumulation per cycle divided by the expected cycle length. In later textbooks this is the renewal–reward theorem, the standard tool for computing long-run averages of regenerative systems in queueing, inventory and reliability models.

Timeline. Doob (1948) proved the strong law nt/t→1/μ1n_t/t \to 1/\mu_1nt​/t→1/μ1​ for renewal counting processes (Trans. AMS 63). Smith (1955) defined cumulative processes by conditions (C1)–(C2) and proved Theorem 7 under a first-moment condition on the variation per cycle, together with central limit theorems (Theorems 9–10) for the same processes.

Setting

Fix a probability space (Ω,F,P)(\Omega, \mathcal F, P)(Ω,F,P).

A renewal process is a sequence t1,t2,…t_1, t_2, \dotst1​,t2​,… of independent, identically distributed, non-negative random variables that are not zero with probability one, i.e. P{t1=0}<1P\{t_1 = 0\} < 1P{t1​=0}<1. Write μ1=Et1\mu_1 = E t_1μ1​=Et1​. In this mission the delay is t0=0t_0 = 0t0​=0, the regeneration epochs are Tk=t0+t1+⋯+tkT_k = t_0 + t_1 + \dots + t_kTk​=t0​+t1​+⋯+tk​, and the counting variable

nt=#{k≥0:Tk≤t}n_t = \#\{k \ge 0 : T_k \le t\}nt​=#{k≥0:Tk​≤t}

is the number of regenerations in [0,t][0, t][0,t] (the paper's "greatest integer kkk such that Tk−1≤tT_{k-1} \le tTk−1​≤t"). The forward variable Zt=∑i=1nt+1ti−tZ_t = \sum_{i=1}^{n_t+1} t_i - tZt​=∑i=1nt​+1​ti​−t measures how far ttt is from the regeneration epoch Tnt+1T_{n_t+1}Tnt​+1​.

Let w=(wt)w = (w_t)w=(wt​) be a real-valued process with w0=0w_0 = 0w0​=0. Its cycle increments are yn=wTn−wTn−1y_n = w_{T_n} - w_{T_{n-1}}yn​=wTn​​−wTn−1​​, n≥1n \ge 1n≥1, and κ1=Ey1\kappa_1 = E y_1κ1​=Ey1​. On a path of bounded variation on every finite interval, the variation process w~t=∫0t∣dwt∣\tilde w_t = \int_0^t |\mathrm d w_t|w~t​=∫0t​∣dwt​∣ is the total variation of the path over [0,t][0, t][0,t]; the cycle variations are y~n=w~Tn−w~Tn−1\tilde y_n = \tilde w_{T_n} - \tilde w_{T_{n-1}}y~​n​=w~Tn​​−w~Tn−1​​, and κ~p=Ey~1 p\tilde\kappa_p = E \tilde y_1^{\,p}κ~p​=Ey~​1p​.

The process www is a cumulative process when

  • (C1) y1,y2,…y_1, y_2, \dotsy1​,y2​,… are independent and identically distributed, and
  • (C2) with probability one, www is of bounded variation in every finite interval.

Formalization targets

Goal: Theorem 7 (the ergodic theorem)

If www is a cumulative process, μ1<∞\mu_1 < \inftyμ1​<∞, κ~1<∞\tilde\kappa_1 < \inftyκ~1​<∞, t0=0t_0 = 0t0​=0 and w0=0w_0 = 0w0​=0, then

lim⁡t→∞wtt=κ1μ1with probability one.\lim_{t \to \infty} \frac{w_t}{t} = \frac{\kappa_1}{\mu_1} \quad \text{with probability one.}t→∞lim​twt​​=μ1​κ1​​with probability one.

The statement fixes no rate and no constant; it is the exact limit of the paper.

Milestones, in the order the proof uses them

  1. Lemma 7. For i.i.d. xnx_nxn​ with E∣xn∣p<∞E|x_n|^p < \inftyE∣xn​∣p<∞, p>0p > 0p>0: xn/n1/p→0x_n / n^{1/p} \to 0xn​/n1/p→0 with probability one.
  2. Renewal strong law (proof of Lemma 8): nt/t→1/μ1n_t / t \to 1/\mu_1nt​/t→1/μ1​ with probability one when μ1<∞\mu_1 < \inftyμ1​<∞.
  3. Lemma 8. If t0=0t_0 = 0t0​=0, μ1<∞\mu_1 < \inftyμ1​<∞ and κ~p<∞\tilde\kappa_p < \inftyκ~p​<∞ (p>0p > 0p>0), then
lim⁡t→∞wt+Zt−wtt1/p=0with probability one.\lim_{t \to \infty} \frac{w_{t+Z_t} - w_t}{t^{1/p}} = 0 \quad \text{with probability one.}t→∞lim​t1/pwt+Zt​​−wt​​=0with probability one.
  1. (5·3·4). ∑i=1nt+1yi/(nt+1)→κ1\sum_{i=1}^{n_t+1} y_i / (n_t + 1) \to \kappa_1∑i=1nt​+1​yi​/(nt​+1)→κ1​ with probability one.
  2. The limit along regeneration epochs. wt+Zt/t→κ1/μ1w_{t+Z_t} / t \to \kappa_1/\mu_1wt+Zt​​/t→κ1​/μ1​ with probability one.

Significance

The ergodic theorem turns a question about a whole sample path into a question about one cycle. Once a system is shown to regenerate, its long-run average cost, throughput or utilization is the ratio of two one-cycle expectations, which can often be computed in closed form. This is how long-run averages are evaluated for M/G/1M/G/1M/G/1-type queues, (s,S)(s,S)(s,S) inventory policies and alternating-renewal reliability models, and it is the first step in average-cost analyses of regenerative Markov decision processes. Lemma 8, with p>1p > 1p>1, also controls the error term in Smith's central limit theorem for cumulative processes.

The result is classical and proved. As far as the platform and Mathlib are concerned, it is not formalized: Mathlib has the strong law of large numbers for i.i.d. sequences (ProbabilityTheory.strong_law_ae) but no renewal counting process, no renewal strong law, and no renewal–reward theorem. A complete development gives machine-checked versions of all three, stated for real time and for cycle lengths that may vanish with positive probability.

Difficulty

The obvious argument writes wt/tw_t / twt​/t as (∑i≤ntyi/nt)⋅(nt/t)\big(\sum_{i \le n_t} y_i / n_t\big) \cdot (n_t / t)(∑i≤nt​​yi​/nt​)⋅(nt​/t) and applies the strong law to each factor. This fails in two places. First, wtw_twt​ is not a sum of whole cycles: the cycle containing ttt is only partly completed, and its contribution has to be shown negligible. The increment yny_nyn​ does not control it, because www may oscillate inside a cycle and return; it is the variation y~n\tilde y_ny~​n​ that bounds the partial contribution, which is why the hypothesis is on κ~1\tilde\kappa_1κ~1​ rather than on κ1\kappa_1κ1​. Second, the strong law is a statement about a deterministic index n→∞n \to \inftyn→∞, while here the index ntn_tnt​ is random and depends on the cycle lengths, which need not be independent of the increments. Transferring almost-sure convergence through the random index requires nt→∞n_t \to \inftynt​→∞ almost surely, which in turn uses P{t1=0}<1P\{t_1 = 0\} < 1P{t1​=0}<1.

Formalization scope

Time is real: every limit is along t→∞t \to \inftyt→∞ in R\mathbb RR (Filter.atTop on ℝ), and "with probability one" is an almost-everywhere statement placed outside the limit. Random variables are real-valued. The renewal process is the structure IsRenewalProcess (cycle lengths measurable, mutually independent, identically distributed, non-negative everywhere, P{t1=0}<1P\{t_1 = 0\} < 1P{t1​=0}<1); t0=0t_0 = 0t0​=0 and w0=0w_0 = 0w0​=0 are explicit hypotheses holding at every sample point. Moments are Bochner integrals guarded by integrability hypotheses: μ1<∞\mu_1 < \inftyμ1​<∞ is integrability of t1t_1t1​, κ~p<∞\tilde\kappa_p < \inftyκ~p​<∞ is integrability of y~1 p\tilde y_1^{\,p}y~​1p​. The positivity μ1>0\mu_1 > 0μ1​>0 is never assumed; it follows from P{t1=0}<1P\{t_1 = 0\} < 1P{t1​=0}<1.

The count ntn_tnt​ is a cardinality; on the null event where infinitely many epochs lie below ttt it takes the value 000, and finiteness is never assumed. The variation process is Mathlib's eVariationOn of the path over [0,t][0, t][0,t], set to 000 on paths that are not of bounded variation, exactly as the paper prescribes.

Condition (C1) is read literally: only the increments yny_nyn​ are i.i.d., and no independence between cycle lengths and increments is assumed. In addition the cycle variations y~n\tilde y_ny~​n​ are assumed identically distributed; the paper's notation κ~p\tilde\kappa_pκ~p​ presupposes this. The statement is not trivialized by a stronger model: taking www to be a Lévy process, or a deterministic multiple of ttt, would make the theorem immediate and is not what is formalized.

Two misprints of the paper are corrected in the formal statements and kept in the verbatim milestone text: (5·3·4) prints the limit as ν1\nu_1ν1​, which must be κ1\kappa_1κ1​; and the proof bound (5·3·1) uses y~nt+1\tilde y_{n_t+1}y~​nt​+1​ alone, where the interval [t,t+Zt][t, t+Z_t][t,t+Zt​] also meets cycle ntn_tnt​.

Needed infrastructure: a renewal counting process in continuous time with its strong law; a lemma that almost-sure convergence of a sequence passes to a random index tending to infinity; the comparison ∣wb−wa∣≤w~b−w~a|w_b - w_a| \le \tilde w_b - \tilde w_a∣wb​−wa​∣≤w~b​−w~a​ for paths of bounded variation. All three are reusable well beyond this mission (renewal–reward arguments, regenerative simulation, average-cost Markov decision processes). Contributions of any milestone, and of these general lemmas as separate theorems, are welcome.

Selected references

  • W. L. Smith, Regenerative stochastic processes, Proceedings of the Royal Society of London, Series A 232(1188):6–31, 1955. https://doi.org/10.1098/rspa.1955.0198
  • J. L. Doob, Renewal theory from the point of view of the theory of probability, Transactions of the American Mathematical Society 63(3):422–438, 1948. https://doi.org/10.1090/S0002-9947-1948-0025098-8
9 thms1 active userReviewed
Algorithmic Game TheoryAnalysisProbability·Captain: mikedeng1

Values of Large Games II: Oceanic Games 2: Oceanic Values Are Limits of Values of Finite Weighted Majority GamesResearch Paper

Motivation

Weighted voting models assign a coalition a win when its combined vote reaches a quota. The Shapley value measures a player's contribution by averaging the change it makes when added to every possible predecessor coalition. This is straightforward to define for a finite list of voters, but a voting body may contain a few major holders and a very large population of individually small holders. A calculation made for a finite approximation should then have a stable meaning as the small holdings are divided more finely. Milnor and Shapley's oceanic game gives that question a precise form and asks whether its major-player values agree with the limits of finite weighted majority games. Milnor and Shapley, RAND RM-2649 (1961).

The question matters whenever a finite voting model is used to represent a diffuse electorate or ownership base. Without a continuity result, the measured power of a major voter could depend on how an otherwise identical mass of minor votes was artificially split. The memo's Theorem 1 establishes the required stability under an explicit small-weight condition. The result is a known theorem from the 1961 memorandum; the task here is its Lean formalization, including the mathematical objects that make its statement meaningful.

Setting

Fix a finite set M={1,…,m}M=\{1,\ldots,m\}M={1,…,m} of major players, with nonnegative weights wiw_iwi​. The ocean is a continuum of individually insignificant voters represented by the unit interval I=[0,1]I=[0,1]I=[0,1], with total weight α>0\alpha>0α>0. A coalition's vote is its major-player weight plus α\alphaα times the Lebesgue measure of its oceanic part. It wins when that vote reaches a quota c≥0c\ge0c≥0. The formal development uses Fin m for MMM and real weights; the underlying voting rule is the one in §2 of the memorandum. Milnor and Shapley, §2.

To assign a value to a major player, insert each major player independently and uniformly into the ordered ocean. A position vector x=(x1,…,xm)x=(x_1,\ldots,x_m)x=(x1​,…,xm​) lies in the cube ImI^mIm. Write P(t)={j∈M:xj<t}P(t)=\{j\in M:x_j<t\}P(t)={j∈M:xj​<t} and w(S)=∑j∈Swjw(S)=\sum_{j\in S}w_jw(S)=∑j∈S​wj​. Major player iii is pivotal when the predecessor weight is below the quota and adding iii reaches it, in the memo's weak-boundary form

w(P(xi))+αxi≤c≤w(P(xi))+wi+αxi.w(P(x_i))+\alpha x_i\le c\le w(P(x_i))+w_i+\alpha x_i.w(P(xi​))+αxi​≤c≤w(P(xi​))+wi​+αxi​.

The oceanic major-player value ϕi\phi_iϕi​ is the probability of this event. Since the insertion positions are uniform, it is also the mmm-dimensional Lebesgue volume of the corresponding set Ai⊆ImA_i\subseteq I^mAi​⊆Im. Equalities at boundary positions have zero volume. Milnor and Shapley, (2.3)–(2.4), pp. 4–5.

At stage ℓ\ellℓ, the finite approximation has the same mmm major players, followed by nℓn_\ellnℓ​ minor players with nonnegative weights aj,ℓa_{j,\ell}aj,ℓ​. Its quota and major weights are cℓc_\ellcℓ​ and wi,ℓw_{i,\ell}wi,ℓ​. A coalition has value one if its weight is at least cℓc_\ellcℓ​, and zero otherwise. The finite-game value ϕi,ℓ\phi_{i,\ell}ϕi,ℓ​ is the Shapley value of that coalition function. The published Shapley-value definition is reused for this general finite-game object; only the quota game and its oceanic limit are defined for this mission. Milnor and Shapley, Appendix (A.1), (A.4).

Formalization targets

Theorem 1: continuity of major-player values

The principal target says that if quotas and major weights converge, the total minor weight tends to α\alphaα, and every minor weight becomes small, then each major-player value converges:

cℓ→c,wi,ℓ→wi,∑jaj,ℓ→α>0,max⁡jaj,ℓ→0⟹ϕi,ℓ→ϕi.c_\ell\to c,\quad w_{i,\ell}\to w_i,\quad \sum_j a_{j,\ell}\to\alpha>0,\quad \max_j a_{j,\ell}\to0 \quad\Longrightarrow\quad \phi_{i,\ell}\to\phi_i.cℓ​→c,wi,ℓ​→wi​,j∑​aj,ℓ​→α>0,jmax​aj,ℓ​→0⟹ϕi,ℓ​→ϕi​.

This is Theorem 1, equations (3.1)–(3.2), of the memorandum. Its conclusion refers to the pivotal-probability definition of ϕi\phi_iϕi​ above. Milnor and Shapley, Theorem 1, p. 6.

Supporting statements

The milestone list follows three statements present in the source: equation (3.4) partitions AiA_iAi​ by the predecessor set SSS; equations (3.5)–(3.6) give the volume of each part as a one-variable integral with clamped endpoints; and Appendix (A.1)–(A.3) states the finite-game limit as a sum of those integrals. Their shared expression is

Li(c,w,α)=∑S⊆M∖{i}∫[0,1]∩[(c−w(S)−wi)/α,(c−w(S))/α]t∣S∣(1−t)m−∣S∣−1 dt.L_i(c,w,\alpha)=\sum_{S\subseteq M\setminus\{i\}}\int_{[0,1]\cap[(c-w(S)-w_i)/\alpha,(c-w(S))/\alpha]} t^{|S|}(1-t)^{m-|S|-1}\,dt.Li​(c,w,α)=S⊆M∖{i}∑​∫[0,1]∩[(c−w(S)−wi​)/α,(c−w(S))/α]​t∣S∣(1−t)m−∣S∣−1dt.

The appendix's limit statement concerns LiL_iLi​; Theorem 1 concerns ϕi\phi_iϕi​. Both are separate targets in the formalization. Milnor and Shapley, §3 and Appendix.

Significance

The theorem makes the major-player value independent of a particular fine division of the minor vote, provided the total minor weight and the other parameters converge as stated. The same oceanic game can therefore stand for many finite approximating sequences. The integral expression also gives a precise comparison point between finite Shapley values and the geometric definition by pivotal volume. Milnor and Shapley, §1 and Theorem 1.

The mathematical result is proved in the cited memorandum, and its appendix recapitulates a limit theorem from the preceding work in the series. The formalization work is to state and prove these known claims over Lean's finite index types, Lebesgue measure, integrals, and limits. Reusable components include the quota-game characteristic function and the interface between a finite Shapley value and a changing number of minor players. The mission does not claim that the historical theorem is open, nor that these target statements already have machine-checked proofs.

Difficulty

The number of minor players changes with ℓ\ellℓ, so the finite Shapley sum does not have a fixed set of coalitions. Merely taking a limit term by term in that sum does not justify its limit: the number and weights of the terms also change. The condition that the largest minor weight tends to zero controls a triangular family of games, while the oceanic definition is a measure of a geometric pivotal event. Matching the two descriptions requires care at the quota boundaries and at intervals that may be empty. These are the central issues represented by the appendix milestone and by equations (3.4)–(3.6). Milnor and Shapley, §3 and Appendix.

Formalization scope

The Lean model uses Fin m for major players and Fin (n l) for minor players at stage lll, concatenated with majors first. It uses a real-valued quota characteristic function, the published finite Shapley-value definition, and product Lebesgue volume on the cube [0,1]m[0,1]^m[0,1]m for uniform insertion positions. The predecessor relation is strict, xj<xix_j<x_ixj​<xi​, exactly as in §2; the complementary cell relation is weak. The pivotal inequalities follow (2.4), and the oceanic value is defined from their volume. Defining it as LiL_iLi​ would make the comparison demanded by Theorem 1 empty, so that equality remains a theorem-level obligation.

The source defines oceanic games with c≥0c\ge0c≥0, wi≥0w_i\ge0wi​≥0, and α>0\alpha>0α>0. The finite approximants are read as weighted majority games with nonnegative weights; that condition is explicit in Lean. No sign condition is imposed on the stage quotas cℓc_\ellcℓ​, since (3.2) states none (only the limit satisfies c≥0c\ge0c≥0). The statement does not impose the footnote's upper bound c≤w(M)+αc\le w(M)+\alphac≤w(M)+α because Theorem 1 only concerns major-player values, which are zero in a null game. An empty minor list is permitted at an early stage. Rather than taking the maximum of an empty list, Lean says that for each ε>0\varepsilon>0ε>0, all minor weights are eventually at most ε\varepsilonε; with nonnegative weights this is the paper's vanishing-maximum condition. A positive limiting ocean weight also excludes an eventually empty minor population.

The integral in (A.3) is a set integral over the stated intersection of closed intervals, so an empty intersection contributes zero. For (3.5), the endpoints are clamped to [0,1][0,1][0,1]. The condition S⊆M∖{i}S\subseteq M\setminus\{i\}S⊆M∖{i} ensures m−∣S∣−1m-|S|-1m−∣S∣−1 is nonnegative before it is represented as a natural-number exponent. Contributions that strengthen the measure-theoretic partition, the cell-volume computation, or the varying-population finite limit are within scope; a proof may use other intermediate lemmas while keeping the sourced target statements unchanged.

Selected references

  • John W. Milnor and Lloyd S. Shapley, Values of Large Games, II: Oceanic Games, RAND Research Memorandum RM-2649, 1961. RAND publication page.
  • Lloyd S. Shapley and Norman Shapiro, Values of Large Games, I: A Limit Theorem, RAND Research Memorandum RM-2648, 1960. RAND publication page.
8 thms1 active userReviewed
PreviousPage 41 of 54Next

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paper
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me