On Synchronous, Asynchronous, and Randomized Best-Response Schemes for Stochastic Nash Games 3: Asynchronous Inexact Best Response with Delays Reaches an ϵ-NE in Explicitly Bounded SG StepsResearch Paper
Motivation
Many equilibrium problems in operations research have the form of a stochastic Nash game: each of players minimizes an expected cost that depends on the strategies of the others, and the expectation can only be sampled. The paper's introduction lists applications in generation-capacity and power markets and in communication networks (Abada, de Maere d'Aertrycke and Smeers, 2017; Başar, 2007). A natural way to compute an equilibrium is a best-response scheme: each player repeatedly re-optimizes against the current strategies of its rivals. In a large network the players cannot synchronize. They update at different times and see their rivals' strategies with delays.
Lei, Shanbhag, Pang and Sen (arXiv:1704.04578v2; Mathematics of Operations Research, 2020) analyze inexact proximal best-response schemes in three regimes: synchronous, randomized and asynchronous. This mission formalizes the asynchronous one (§5 and Appendix C). It adapts the partially asynchronous iterations of Bertsekas and Tsitsiklis (1989) to stochastic games with inexact, sampled best responses, and it gives an explicit bound on the number of stochastic gradient steps each player needs.
Setting
Player chooses in a nonempty compact convex set . A profile is . Player 's cost is , which is convex and twice continuously differentiable in a neighbourhood of . A sampling oracle returns , whose second moment is at most (Assumption 1). A Nash equilibrium is a profile in which each minimizes over .
For , the proximal best response of player to a profile is
The curvature constants and define the matrix with and . Assumption 5 (strict diagonal dominance) asks . It makes , the maximum absolute row sum, smaller than one.
The asynchronous scheme (Algorithm 3) runs on a deterministic schedule. At time the players in update. Player sees the outdated profile and computes with
The other players keep their strategies. Assumption 4 requires every player to update at least once in every consecutive times, and every delay to be at most . The accuracy is , where counts player 's updates so far. The inexact response is computed by projected stochastic gradient steps (43), with step size and . Write , and .
Formalization targets
Goal: Theorem 3, (45)
Let , , , and
where . After major iterations, , and player has taken at most
projected gradient steps.
Milestones
In the order the proof uses them:
- the -contraction (5) of the proximal best response;
- the fixed-point identity ;
- under Assumption 5;
- its expected -norm form (40), ;
- the one-step recursion (C.2) and the delay step (C.3) of Appendix C;
- Lemma 2, ;
- Lemma 7, the rate
- Lemma 8, the mean-square error of the inner stochastic approximation loop;
- the summation bound (30).
Significance
Theorem 3 makes the cost of asynchrony explicit. Bounded delays and a bounded update window enter only through the rate per window, . The overall effort stays polynomial in , and the paper's Table 1 summarizes the exponent as for a suitable choice of . The result connects classical partially asynchronous fixed-point theory with the sample-complexity analysis of stochastic approximation. Lemma 7 holds for any inexact solver that achieves the accuracy (38), not only for the stochastic gradient loop, so it applies to other subproblem solvers as well.
The paper proves these results by hand. As far as is known, no part of them has been machine-checked: there is no formal treatment of stochastic Nash games, of proximal best-response maps, or of asynchronous iterations with delays in Mathlib or on this platform. A formalization would check the induction over windows in Appendix C, whose index bookkeeping has several printed slips. It would also make precise which properties of the delays the argument needs.
Difficulty
The obvious argument does not carry over from the synchronous case. There, one step of the scheme contracts the whole error vector by in one norm. Here a step updates only some players, and each of them uses information up to steps old. No single step contracts anything. The proof has to show that every window of steps contracts the worst-case expected error by a factor , with the delays absorbed into the root . This requires a nested induction with case distinctions on the window position (Appendix C). The analysis also mixes three layers: a deterministic contraction of the exact best response; conditional expectations, Jensen's inequality and adaptedness for the inexact responses; and an stochastic-approximation bound for the inner loop, which in turn needs strong convexity, the first-order optimality conditions and nonexpansiveness of the projection.
Formalization scope
Players are indexed by Fin N. Player 's space is EuclideanSpace ℝ (Fin (n i)), and costs are functions of the whole profile: is f i (Function.update y i z). The projection is any map satisfying the published predicate SpectralProjGrad.Shared.IsProjOnto. The proximal best response is a map constrained by an argmin predicate on . and are Rayleigh-quotient infima and suprema of second Fréchet derivatives, and is the maximum absolute row sum. The probability space carries a filtration to which the iterates are adapted. The inner-loop -algebras are parameters satisfying the inclusions the proof uses. Conditional expectations are Mathlib's condExp.
Statement repairs and implicit hypotheses, each recorded in the item it affects:
- Deterministic delays. Assumption 4(c) allows random delays, but step (C.5) of the proof bounds an expectation at a random past time by the maximum over past times. That step is valid only when the delays do not depend on the iterates. The delays here are deterministic functions with .
- Oracle second moment. Lemma 8 prints for , and its hypothesis forces zero noise. The bound that the proof uses replaces it.
- Regularity. Joint regularity of is assumed, which the mixed Hessian blocks in need, and is nonempty.
- Constants. Theorem 3 assumes and . Its "" is read as , as in Lemma 7.
- Step count. Theorem 3 counts the steps over player 's update times; the proof's bound is for the larger sum over all times.
The special case , of Theorem 3, Corollary 3 (cyclic updates) and Theorem 4 are not stated.
The goal cannot be met trivially. The schedule is a parameter, not "every player at every step". The sampled gradients may be genuinely random, since the second-moment hypothesis is an inequality. is tied to by its argmin property, and is a Nash equilibrium. Each expectation in a conclusion is of a bounded measurable quantity, so it cannot vanish by non-integrability.
The development needs the following:
- the contraction (5), which the paper only cites from Facchinei and Pang (2009, §12.6.1) and which uses the mean-value theorem along segments in ;
- strong-convexity optimality conditions on convex sets;
- nonexpansiveness of Euclidean projections;
- conditional Jensen and tower arguments;
- an integer-index induction over windows.
The projection and strong-convexity lemmas, and (5) itself, are reusable for the synchronous and randomized missions of this series. Proofs of individual milestones, or of general Mathlib-level facts such as the first-order optimality condition over a convex set, are welcome.
Selected references
- J. Lei, U. V. Shanbhag, J.-S. Pang, S. Sen, On Synchronous, Asynchronous, and Randomized Best-Response Schemes for Stochastic Nash Games, arXiv:1704.04578v2, 2018; Mathematics of Operations Research, 2020, https://doi.org/10.1287/moor.2018.0986. https://arxiv.org/abs/1704.04578v2
- D. P. Bertsekas, J. N. Tsitsiklis, Parallel and Distributed Computation: Numerical Methods, Prentice Hall, 1989 (reference [9] of the paper).
- F. Facchinei, J.-S. Pang, Nash equilibria: the variational approach, in Convex Optimization in Signal Processing and Communications, Cambridge University Press, 2009 (reference [19] of the paper).
- I. Abada, G. de Maere d'Aertrycke, Y. Smeers, On the multiplicity of solutions in generation capacity investment models with incomplete markets, Mathematical Programming 165(1):5–69, 2017 (reference [1] of the paper).