Prove2Me
Navigate
DiscoverFormalpediaBlogsUsersMomentumMy Missions+
Prove2Me
⌕
Log in

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

Probability

569 missions · 282 completed

Missions

Open287Completed282All569
Dynamic ProgrammingOperations ResearchStochastic Systems·Captain: mikedeng1

Exit Problems for Spectrally Negative Lévy Processes and Applications to (Canadized) Russian Options II: Optimal Stopping for the Perpetual Russian OptionResearch Paper

Motivation

A Russian option is a perpetual American-type claim that pays, when the holder exercises at time τ\tauτ, the maximum of the asset price seen so far, discounted by e−ατe^{-\alpha\tau}e−ατ. It was introduced by Shepp and Shiryaev for the Black–Scholes market (Shepp–Shiryaev 1993), where the underlying log-price is a Brownian motion with drift. Empirical work on asset returns (skewness, heavy tails, downward jumps) motivates replacing the Brownian motion by a Lévy process with negative jumps only. Avram, Kyprianou and Pistorius (2004) solve the Russian optimal stopping problem in that model in closed form, in terms of the scale functions of the process.

Timeline. 1993: Shepp and Shiryaev solve the Russian problem for geometric Brownian motion; Duffie and Harrison give its no-arbitrage price. Graversen and Peskir, and Kyprianou and Pistorius, treat further variants within the Black–Scholes market (the works the paper cites in §6). 2004: Avram, Kyprianou and Pistorius solve it for every spectrally negative Lévy process, covering both unbounded and bounded variation, using the exit problem of the reflected process Y=X‾−XY=\overline X-XY=X−X (their Theorem 1, the subject of the first mission of this series).

Setting

Let (Ω,F,F={Ft}t≥0,P)(\Omega,\mathcal F,\mathbf F=\{\mathcal F_t\}_{t\ge0},\mathbb P)(Ω,F,F={Ft​}t≥0​,P) be a filtered probability space with a right-continuous filtration, and X={Xt, t≥0}X=\{X_t,\ t\ge0\}X={Xt​, t≥0} a spectrally negative Lévy process for F\mathbf FF: X0=0X_0=0X0​=0, càdlàg paths with no positive jumps, independent and stationary increments with Xs+t−XsX_{s+t}-X_sXs+t​−Xs​ independent of Fs\mathcal F_sFs​, and paths that are not monotone. The standing assumption of the paper is that XXX has unbounded variation, or bounded variation and a Lévy measure absolutely continuous with respect to Lebesgue measure.

The Laplace exponent is ψ(θ)=log⁡E[eθX1]\psi(\theta)=\log\mathbb E[e^{\theta X_1}]ψ(θ)=logE[eθX1​], and Φ(q)\Phi(q)Φ(q) is the largest root of ψ(θ)=q\psi(\theta)=qψ(θ)=q. For q≥0q\ge0q≥0 the qqq-scale function W(q):R→[0,∞)W^{(q)}:\mathbb R\to[0,\infty)W(q):R→[0,∞) is the unique function that vanishes on (−∞,0](-\infty,0](−∞,0], is continuous on (0,∞)(0,\infty)(0,∞), and satisfies ∫0∞e−θxW(q)(x) dx=(ψ(θ)−q)−1\int_0^\infty e^{-\theta x}W^{(q)}(x)\,dx=(\psi(\theta)-q)^{-1}∫0∞​e−θxW(q)(x)dx=(ψ(θ)−q)−1 for θ>Φ(q)\theta>\Phi(q)θ>Φ(q). Then Z(q)(x)=1+q∫−∞xW(q)(z) dzZ^{(q)}(x)=1+q\int_{-\infty}^xW^{(q)}(z)\,dzZ(q)(x)=1+q∫−∞x​W(q)(z)dz. The tilted scale functions Wv(p)W_v^{(p)}Wv(p)​ are those of the exponent ψv(θ)=ψ(θ+v)−ψ(v)\psi_v(\theta)=\psi(\theta+v)-\psi(v)ψv​(θ)=ψ(θ+v)−ψ(v).

Fix r≥0r\ge0r≥0 with ψ(1)=r\psi(1)=rψ(1)=r (the risk-neutral condition), and let P1\mathbb P^1P1 be the Esscher measure, dP1/dP∣Ft=eXt−rtd\mathbb P^1/d\mathbb P|_{\mathcal F_t}=e^{X_t-rt}dP1/dP∣Ft​​=eXt​−rt. For z≥0z\ge0z≥0, under P−z1\mathbb P^1_{-z}P−z1​ the process starts at −z-z−z with running maximum X‾t=max⁡{0,sup⁡u≤tXu}\overline X_t=\max\{0,\sup_{u\le t}X_u\}Xt​=max{0,supu≤t​Xu​}, and the reflected process Y=X‾−XY=\overline X-XY=X−X starts at Y0=zY_0=zY0​=z. The passage time is τk=inf⁡{t≥0:Yt∉[0,k)}\tau_k=\inf\{t\ge0:Y_t\notin[0,k)\}τk​=inf{t≥0:Yt​∈/[0,k)}. Fix α>0\alpha>0α>0 and put q=α+rq=\alpha+rq=α+r.

The Russian optimal stopping problem (28) is

wR(z)=sup⁡τ E−z1[e−ατ+Yτ],w^R(z)=\sup_\tau\ \mathbb E^1_{-z}\big[e^{-\alpha\tau+Y_\tau}\big],wR(z)=τsup​ E−z1​[e−ατ+Yτ​],

the supremum over all P1\mathbb P^1P1-almost surely finite F\mathbf FF-stopping times. The option price is Vr(M0,S0)=S0 wR(log⁡(M0/S0))V_r(M_0,S_0)=S_0\,w^R(\log(M_0/S_0))Vr​(M0​,S0​)=S0​wR(log(M0​/S0​)).

Formalization targets

Goal: Theorem 2

With the optimal level (30) and the candidate value

κ∗=inf⁡{x: Z(q)(x)≤qW(q)(x)},u(z)=ezZ(q)(κ∗−z),\kappa^*=\inf\{x:\ Z^{(q)}(x)\le qW^{(q)}(x)\},\qquad u(z)=e^zZ^{(q)}(\kappa^*-z),κ∗=inf{x: Z(q)(x)≤qW(q)(x)},u(z)=ezZ(q)(κ∗−z),

for every z≥0z\ge0z≥0,

wR(z)=u(z)=E−z1[e−ατκ∗+Yτκ∗],w^R(z)=u(z)=\mathbb E^1_{-z}\big[e^{-\alpha\tau_{\kappa^*}+Y_{\tau_{\kappa^*}}}\big],wR(z)=u(z)=E−z1​[e−ατκ∗​+Yτκ∗​​],

and τκ∗\tau_{\kappa^*}τκ∗​ is a P1\mathbb P^1P1-a.s. finite F\mathbf FF-stopping time.

Milestones

  1. Remark 4: W(u)(x)=evxWv(u−ψ(v))(x)W^{(u)}(x)=e^{vx}W_v^{(u-\psi(v))}(x)W(u)(x)=evxWv(u−ψ(v))​(x).
  2. Lemma 1: Z(q)(x)/W(q)(x)→q/Φ(q)Z^{(q)}(x)/W^{(q)}(x)\to q/\Phi(q)Z(q)(x)/W(q)(x)→q/Φ(q) as x→∞x\to\inftyx→∞ (for q≥0q\ge0q≥0, with the paper's convention for 0/Φ(0)0/\Phi(0)0/Φ(0)).
  3. Remark 3: Wv(0+)=0W_v(0+)=0Wv​(0+)=0 if and only if XXX has unbounded variation.
  4. Corollary 1, (29): the value of stopping at τk\tau_kτk​,
E−z1(e−ατk+Yτk)=ez(Z(q)(k−z)+Z(q)(k)−qW(q)(k)W(q)′(k)−W(q)(k)W(q)(k−z)).\mathbb E^1_{-z}\big(e^{-\alpha\tau_k+Y_{\tau_k}}\big)=e^z\Big(Z^{(q)}(k-z)+\frac{Z^{(q)}(k)-qW^{(q)}(k)}{W^{(q)\prime}(k)-W^{(q)}(k)}W^{(q)}(k-z)\Big).E−z1​(e−ατk​+Yτk​​)=ez(Z(q)(k−z)+W(q)′(k)−W(q)(k)Z(q)(k)−qW(q)(k)​W(q)(k−z)).
  1. Lemma 2 (i): for q>rq>rq>r, f=Z(q)−qW(q)f=Z^{(q)}-qW^{(q)}f=Z(q)−qW(q) decreases on [0,∞)[0,\infty)[0,∞) to −∞-\infty−∞.
  2. Lemma 2 (ii): κ∗=0\kappa^*=0κ∗=0 if W(q)(0+)≥q−1W^{(q)}(0+)\ge q^{-1}W(q)(0+)≥q−1; otherwise κ∗>0\kappa^*>0κ∗>0 is the unique root of fff.
  3. The stopped process e−α(t∧τκ∗)u(Yt∧τκ∗)e^{-\alpha(t\wedge\tau_{\kappa^*})}u(Y_{t\wedge\tau_{\kappa^*}})e−α(t∧τκ∗​)u(Yt∧τκ∗​​) is a P1\mathbb P^1P1-martingale.
  4. E−z1[e−αt+YtZ(q)(κ∗−Yt)]≤ezZ(q)(κ∗−z)\mathbb E^1_{-z}[e^{-\alpha t+Y_t}Z^{(q)}(\kappa^*-Y_t)]\le e^zZ^{(q)}(\kappa^*-z)E−z1​[e−αt+Yt​Z(q)(κ∗−Yt​)]≤ezZ(q)(κ∗−z).
  5. e−αtu(Yt)e^{-\alpha t}u(Y_t)e−αtu(Yt​) is a P1\mathbb P^1P1-supermartingale.

Significance

The result. Theorem 2 gives the price of the perpetual Russian option and its optimal exercise rule for every exponential spectrally negative Lévy market. The rule is to exercise when the ratio of the running maximum to the current price first reaches eκ∗e^{\kappa^*}eκ∗. The level is explicit through scale functions, and it separates the regimes: for bounded variation with W(q)(0+)≥q−1W^{(q)}(0+)\ge q^{-1}W(q)(0+)≥q−1, immediate exercise is optimal. The theorem is the model case of a general method: an optimal stopping problem for a functional of (X,X‾)(X,\overline X)(X,X) is reduced, by a change of measure, to one for the reflected process, and solved by verification. The same method underlies the Canadized Russian option (third mission of the series).

Formalizing it. The result is proved in the paper; it has no machine-checked proof. This mission produces a Lean statement of the full verification theorem, including admissibility of τκ∗\tau_{\kappa^*}τκ∗​. It also states the analytic facts about scale functions that the proof relies on (Lemmas 1, 2 and Remarks 3, 4), which apply to any problem phrased in scale functions.

Difficulty

The obvious route is the classical verification: show that e−αtu(Yt)e^{-\alpha t}u(Y_t)e−αtu(Yt​) is a supermartingale, apply optional stopping, and check equality at τκ∗\tau_{\kappa^*}τκ∗​. The first step fails as a direct Itô computation. In the unbounded-variation case uuu is only C1C^1C1 at κ∗\kappa^*κ∗, and in the bounded-variation case only continuous there. The generator of YYY is nonlocal, so smoothness away from κ∗\kappa^*κ∗ does not control the jump part of the process across the boundary. The equality case needs the exact value of stopping at τk\tau_kτk​ (Corollary 1). That value requires the overshoot of YYY over kkk, which is caused by a downward jump of XXX, and the exit problem of the reflected process. Finally, P1\mathbb P^1P1 is not equivalent to P\mathbb PP on F∞\mathcal F_\inftyF∞​, so passing from P\mathbb PP-facts to P1\mathbb P^1P1-facts is valid only on each Ft\mathcal F_tFt​.

Formalization scope

Conventions committed to in Lean:

  • Time is [0,∞)[0,\infty)[0,∞) (ℝ≥0) and values are real. Random times take values in [0,∞][0,\infty][0,∞] (WithTop ℝ≥0), and the payoff is set to 000 on {τ=∞}\{\tau=\infty\}{τ=∞}, a P1\mathbb P^1P1-null event for admissible τ\tauτ.
  • The spectrally negative Lévy process is a structure: measurable marginals, X0=0X_0=0X0​=0, independent increments, stationary increments, càdlàg paths, no positive jumps, not almost surely monotone. Paths start at 000, are càdlàg, and have no positive jumps for every ω\omegaω, not merely almost surely. Adaptedness, independence of increments from the past, and right-continuity of F\mathbf FF are added for the filtered version.
  • "The usual conditions" are read as right-continuity only; completeness is not imposed. P1\mathbb P^1P1 is typically singular to P\mathbb PP on F∞\mathcal F_\inftyF∞​, so a complete F0\mathcal F_0F0​ would contradict (3).
  • "Unbounded variation" means "not almost surely of bounded variation on compacts". Condition (AC) is stated through jumps: no jump lands in a Lebesgue-null set, almost surely. The standing assumption is "bounded variation implies (AC)".
  • ψ\psiψ is Mathlib's cumulant generating function; "ψ(v)<∞\psi(v)<\inftyψ(v)<∞" is integrability of evX1e^{vX_1}evX1​.
  • W(q)W^{(q)}W(q) is a definite description (choice among functions with the properties of Definition 2) for q≥0q\ge0q≥0, and the series (5) for q<0q<0q<0. Z(q)Z^{(q)}Z(q) integrates over (−∞,x](-\infty,x](−∞,x].
  • P1\mathbb P^1P1 is data (a probability measure Q\mathbb QQ) with Q∣Ft=eXt−rt⋅P∣Ft\mathbb Q|_{\mathcal F_t}=e^{X_t-rt}\cdot\mathbb P|_{\mathcal F_t}Q∣Ft​​=eXt​−rt⋅P∣Ft​​ for all ttt. P−z1\mathbb P^1_{-z}P−z1​ is encoded by the reflected process with prior maximum 000 and starting point −z-z−z.
  • The value function is a supremum in [0,∞][0,\infty][0,∞] of lower Lebesgue integrals. Expectation identities (Corollary 1, the display on p. 230) are stated in [0,∞][0,\infty][0,∞] and thereby assert finiteness.
  • W(q)(0+)W^{(q)}(0+)W(q)(0+) is Function.rightLim, and κ∗\kappa^*κ∗ is the real infimum (30); its nonemptiness is Lemma 2, not a hypothesis. τ0=0\tau_0=0τ0​=0 extends the paper's τk\tau_kτk​, k>0k>0k>0.
  • Readings of informal words: "decreases monotonically" is strict decrease on [0,∞)[0,\infty)[0,∞); "the unique root" is on [0,∞)[0,\infty)[0,∞); Lemma 1 is formalized for q≥0q\ge0q≥0, with 0/Φ(0)0/\Phi(0)0/Φ(0) read as lim⁡θ↓0θ/Φ(θ)\lim_{\theta\downarrow0}\theta/\Phi(\theta)limθ↓0​θ/Φ(θ) as the paper stipulates; Remarks 3 and 4 for real qqq, uuu only. The p. 230 martingale, bound and supermartingale claims are stated under the standing assumption, covering all three cases of the proof.

A trivializing formalization is ruled out: the supremum ranges over every P1\mathbb P^1P1-a.s. finite stopping time of the given filtration (not only passage times, not a smaller filtration), and it is taken in [0,∞][0,\infty][0,∞], where no junk value of an unbounded real supremum can occur.

Infrastructure a complete development needs: Lévy processes on path space, their Laplace exponents and Esscher transforms, scale functions (existence, uniqueness, smoothness under the standing assumption), the reflected process and the exit identity of Theorem 1, optional stopping for continuous-time supermartingales, and Itô/change-of-variables formulas for semimartingales with jumps. The scale-function and Esscher layers are reusable for the other missions of this series and for any fluctuation-theory problem. Contributions to any of these layers, or to the milestones separately, are welcome.

Selected references

  • F. Avram, A. E. Kyprianou, M. R. Pistorius, Exit problems for spectrally negative Lévy processes and applications to (Canadized) Russian options, Ann. Appl. Probab. 14(1), 215–238, 2004. https://doi.org/10.1214/aoap/1075828052
  • L. Shepp, A. N. Shiryaev, The Russian option: reduced regret, Ann. Appl. Probab. 3(3), 631–640, 1993. https://doi.org/10.1214/aoap/1177005715
  • J. Bertoin, Lévy Processes, Cambridge Tracts in Mathematics 121, Cambridge University Press, 1996. ISBN 0-521-56243-0
  • A. E. Kyprianou, Fluctuations of Lévy Processes with Applications, 2nd ed., Springer, 2014. https://doi.org/10.1007/978-3-642-37632-0
17 thms1 active userReviewed
Dynamical SystemsReinforcement LearningStochastic Systems·Captain: mikedeng1

The O.D.E. Method for Convergence of Stochastic Approximation and Reinforcement Learning II: Mean-Square Error under Bounded StepsizesResearch Paper

Motivation

Stochastic approximation is the family of recursive algorithms that locate a zero of a vector field hhh from noisy evaluations of it. Temporal-difference learning, Q-learning, actor–critic methods and stochastic gradient descent are all instances. In practice these algorithms are often run with a constant or bounded, non-vanishing stepsize: the iterate keeps adapting to new data and never freezes, at the price of never converging exactly. The natural question for such a scheme is quantitative: how far from the target does the iterate stay in the long run, and how does that distance scale with the stepsize?

Borkar and Meyn (SIAM J. Control Optim. 38(2), 2000) answer both halves of the question under one set of hypotheses. First, stability of a "fluid" ODE obtained by scaling hhh at infinity implies that the iterates have bounded second moments, with no a priori boundedness or projection assumption. Second, if the ODE x˙=h(x)\dot x = h(x)x˙=h(x) has a globally exponentially stable equilibrium x∗x^*x∗, the asymptotic mean-square error is of the order of the largest stepsize. The first result replaced the usual stochastic-Lyapunov-function verification in reinforcement-learning applications (Section 3 of the paper). This mission formalizes the bounded-stepsize branch of the paper. A companion mission (… I: Stability and Almost-Sure Convergence under Tapering Stepsizes) covers the vanishing-stepsize branch.

Setting

Fix d≥1d \ge 1d≥1 and a Lipschitz vector field h:Rd→Rdh : \mathbb{R}^d \to \mathbb{R}^dh:Rd→Rd. The stochastic approximation recursion (1.1) is

X(n+1)=X(n)+a(n)[h(X(n))+M(n+1)],n≥0,X(n+1) = X(n) + a(n)\big[h(X(n)) + M(n+1)\big], \qquad n \ge 0,X(n+1)=X(n)+a(n)[h(X(n))+M(n+1)],n≥0,

with a deterministic step sequence {a(n)}\{a(n)\}{a(n)} and a noise sequence {M(n)}\{M(n)\}{M(n)} on a probability space (Ω,F,P)(\Omega, \mathcal F, \mathsf P)(Ω,F,P). The associated ODE (1.2) is x˙=h(x)\dot x = h(x)x˙=h(x).

  • The scaled fields are hr(x)=r−1h(rx)h_r(x) = r^{-1} h(rx)hr​(x)=r−1h(rx). Assumption (A1) asks that hhh be Lipschitz, that hr(x)→h∞(x)h_r(x) \to h_\infty(x)hr​(x)→h∞​(x) for every xxx as r→∞r \to \inftyr→∞, and that the origin be an asymptotically stable equilibrium of the fluid ODE x˙=h∞(x)\dot x = h_\infty(x)x˙=h∞​(x).
  • Let Fn=σ(X(0),…,X(n))\mathcal F_n = \sigma(X(0), \dots, X(n))Fn​=σ(X(0),…,X(n)). Assumption (A2) asks that {M(n)}\{M(n)\}{M(n)} be a martingale difference sequence, E[M(n+1)∣Fn]=0\mathsf E[M(n+1) \mid \mathcal F_n] = 0E[M(n+1)∣Fn​]=0, with conditional second moments E[∥M(n+1)∥2∣Fn]≤C0(1+∥X(n)∥2)\mathsf E[\|M(n+1)\|^2 \mid \mathcal F_n] \le C_0(1 + \|X(n)\|^2)E[∥M(n+1)∥2∣Fn​]≤C0​(1+∥X(n)∥2) for a constant C0C_0C0​.
  • Assumption (BS), bounded stepsizes: 0<α‾≤a(n)≤αˉ<10 < \underline\alpha \le a(n) \le \bar\alpha < 10<α​≤a(n)≤αˉ<1 for all nnn, with α‾<αˉ\underline\alpha < \bar\alphaα​<αˉ.
  • The error (2.2) is e(n)=∥X(n)−x∗∥e(n) = \|X(n) - x^*\|e(n)=∥X(n)−x∗∥.

An equilibrium x∗x^*x∗ of (1.2) is globally asymptotically stable if it is Lyapunov stable and attracts every solution. It is globally exponentially asymptotically stable if there are bbb and δ>0\delta > 0δ>0 with ∥x(t)−x∗∥≤b e−δt∥x(0)−x∗∥\|x(t) - x^*\| \le b\,e^{-\delta t}\|x(0) - x^*\|∥x(t)−x∗∥≤be−δt∥x(0)−x∗∥ for every solution.

The proofs compare the iterates with ODE solutions on a time grid: t(n)=∑i<na(i)t(n) = \sum_{i<n} a(i)t(n)=∑i<n​a(i), blocks T(j)=t(m(j))T(j) = t(m(j))T(j)=t(m(j)) of length about TTT, the interpolated path ψ\psiψ of the iterates, its rescaled version ϕj=ψ/r(j)\phi_j = \psi/r(j)ϕj​=ψ/r(j) with r(j)=max⁡(1,∥X(m(j))∥)r(j) = \max(1, \|X(m(j))\|)r(j)=max(1,∥X(m(j))∥), and the ODE solutions ψ^\hat\psiψ^​, ϕ^j\hat\phi_jϕ^​j​ restarted at each block.

Formalization targets

Goal: Theorem 2.3(ii)

Under (A1), (A2) and (BS), if x∗x^*x∗ is a globally asymptotically and globally exponentially asymptotically stable equilibrium of (1.2), there are α∗>0\alpha^* > 0α∗>0 and b2<∞b_2 < \inftyb2​<∞ such that for all 0<αˉ≤α∗0 < \bar\alpha \le \alpha^*0<αˉ≤α∗ and every initial condition X(0)X(0)X(0),

lim sup⁡n→∞E[e(n)2]≤b2 αˉ.\limsup_{n \to \infty} \mathsf E\big[e(n)^2\big] \le b_2\, \bar\alpha .n→∞limsup​E[e(n)2]≤b2​αˉ.

The constant b2b_2b2​ is uniform in the stepsizes and in X(0)X(0)X(0). No rate constant is fixed: only the linear dependence on αˉ\bar\alphaαˉ is asserted.

Milestones

  1. Lemma 4.7. For fixed T>0T > 0T>0 there is C2C_2C2​, independent of the stepsizes, with E[∥ϕj(t)−ϕ^j(t)∥2∣Fm(j)]≤C2αˉ\mathsf E[\|\phi_j(t) - \hat\phi_j(t)\|^2 \mid \mathcal F_{m(j)}] \le C_2\bar\alphaE[∥ϕj​(t)−ϕ^​j​(t)∥2∣Fm(j)​]≤C2​αˉ and E[∥ϕj(t)∥2∣Fm(j)]≤C2\mathsf E[\|\phi_j(t)\|^2 \mid \mathcal F_{m(j)}] \le C_2E[∥ϕj​(t)∥2∣Fm(j)​]≤C2​ on every block.
  2. Theorem 2.1(ii). There are α∗>0\alpha^* > 0α∗>0 and C1C_1C1​ with lim sup⁡nE∥X(n)∥2≤C1\limsup_n \mathsf E\|X(n)\|^2 \le C_1limsupn​E∥X(n)∥2≤C1​ whenever αˉ<α∗\bar\alpha < \alpha^*αˉ<α∗.
  3. Lemma 4.8. For αˉ≤α∗\bar\alpha \le \alpha^*αˉ≤α∗, sup⁡t≥0E∥ψ^(t)−ψ(t)∥2≤C3αˉ\sup_{t \ge 0} \mathsf E\|\hat\psi(t) - \psi(t)\|^2 \le C_3 \bar\alphasupt≥0​E∥ψ^​(t)−ψ(t)∥2≤C3​αˉ.

Significance

The result. Theorem 2.1(ii) says that a stability property of a deterministic ODE at infinity controls a stochastic recursion in mean square, uniformly over initial conditions. Theorem 2.3(ii) turns this into an error bound: with a non-vanishing stepsize the iterate does not converge, but its mean-square distance from x∗x^*x∗ is eventually O(αˉ)O(\bar\alpha)O(αˉ). This is the quantitative justification for constant-stepsize stochastic approximation in reinforcement learning and adaptive control, and it is the starting point of the trade-off between bias (small stepsize) and speed (large stepsize) discussed in Section 2.2 of the paper.

Formalizing it. The results are proved in the paper, partly by sketch: Lemma 4.8's proof refers to "familiar arguments using the Bellman–Gronwall lemma", and the paper writes several bounds as O(αˉ)O(\bar\alpha)O(αˉ). A formal proof makes every constant and its dependence explicit, which the paper's quantifier order leaves partly implicit (see the scope section). No machine-checked proof of these results, or of any stochastic-approximation stability theorem in this generality, is known to exist.

Difficulty

The natural first argument, that the iterates track the ODE and the ODE converges, needs the iterates to be bounded. Bounded iterates are exactly what is being proved, so the argument is circular. The paper breaks the circularity by rescaling: on each block the iterate is divided by r(j)r(j)r(j), so that on large scales it tracks the fluid ODE, whose stability contracts the norm by a fixed factor per block. Making this work in mean square requires conditional second-moment estimates that are uniform in the stepsize and the scale (Lemma 4.7). A second difficulty is that the stepsize does not vanish: the tracking error per block does not go to zero, and the goal's content is precisely that it is of order αˉ\bar\alphaαˉ with a constant that does not depend on αˉ\bar\alphaαˉ.

Formalization scope

  • The state space is EuclideanSpace ℝ (Fin d). ODE solutions are forward solutions: continuous on [0,∞)[0,\infty)[0,∞) with right derivatives. Stability notions are the standard ones, with exponential stability in the form the proof of Theorem 2.3(ii) uses.
  • The filtration is the natural filtration of the iterates. (A2) includes integrability of M(n+1)M(n+1)M(n+1) and ∥M(n+1)∥2\|M(n+1)\|^2∥M(n+1)∥2 so that the conditional expectations are meaningful. Each theorem quantifies over every probability space, every measurable process satisfying (1.1) and (A2), and every deterministic X(0)X(0)X(0).
  • Second moments, suprema and lim sup⁡\limsuplimsup are computed in [0,∞][0,\infty][0,∞], so a bound is a genuine finiteness claim.
  • Constant placement. α∗\alpha^*α∗, C1C_1C1​ and b2b_2b2​ depend only on hhh, h∞h_\inftyh∞​, C0C_0C0​ (and x∗x^*x∗). C2C_2C2​ depends in addition on TTT, and C3C_3C3​ on TTT and X(0)X(0)X(0). None depends on the stepsizes. Theorem 2.3's page text puts b2b_2b2​ after α\alphaα. Read literally, that would allow b2=C1/αˉb_2 = C_1/\bar\alphab2​=C1​/αˉ and make the goal a restatement of Theorem 2.1(ii). The formal goal fixes b2b_2b2​ before the stepsizes and the initial condition, as the proof (16C3αˉ16C_3\bar\alpha16C3​αˉ) gives. α∗\alpha^*α∗ is existential in Theorem 2.3(ii) and Lemma 4.8 rather than tied to Theorem 2.1(ii)'s witness.
  • The proof objects ψ^\hat\psiψ^​, ϕ^j\hat\phi_jϕ^​j​ are characterized by predicates (initial value, continuity, ODE on the block). Lemmas 4.7 and 4.8 hold for every function satisfying them.
  • Not included: Theorem 2.3(i) (its proof chooses a radius depending on αˉ\bar\alphaαˉ), Theorem 2.4 and the Markov-chain results of Section 4.3, and the asynchronous extension (Theorem 2.5). The deterministic ODE lemmas 4.1–4.4 belong to the companion mission.

Useful infrastructure, reusable beyond this mission: Bellman–Gronwall inequalities in discrete time (Lemma 4.3 of the paper), stability of scaled ODEs, and conditional second-moment estimates for martingale-difference-driven recursions. Contributions of any of these as separate lemmas are welcome.

Selected references

  • V. S. Borkar and S. P. Meyn, The O.D.E. Method for Convergence of Stochastic Approximation and Reinforcement Learning, SIAM J. Control Optim. 38(2):447–469, 2000. https://doi.org/10.1137/S0363012997331639
  • M. Benaïm, Dynamics of stochastic approximation algorithms, Séminaire de Probabilités XXXIII, Lecture Notes in Math. 1709, Springer, 1999. https://doi.org/10.1007/BFb0096509
  • H. J. Kushner and G. G. Yin, Stochastic Approximation and Recursive Algorithms and Applications, 2nd ed., Springer, 2003. https://doi.org/10.1007/b97441
7 thms1 active userReviewed
Dynamic ProgrammingOperations ResearchStochastic Systems·Captain: mikedeng1

On the Optimal Dividend Problem for a Spectrally Negative Lévy Process II: Optimality of the Double-Barrier Strategy at d* with Bail-Out LoansResearch Paper

Dividends, ruin and bail-out loans

An insurance company's surplus in the Cramér–Lundberg model grows linearly with premiums and falls by claims arriving as a compound Poisson process. When premium income exceeds the expected claims, the surplus drifts to infinity. De Finetti (1957) proposed that the surplus above a barrier should instead be paid to shareholders as dividends, and the resulting optimal dividend problem — maximize the expected discounted dividends — is a central problem of risk theory. A barrier policy, however, drives the surplus below zero with probability one. Harrison and Taylor (1978) and Løkka and Zervos studied, for Brownian motion, a variant with bail-out loans: ruin is forbidden, and the shareholders must inject capital whenever the surplus would become negative, at a cost φ>1\varphi>1φ>1 per unit.

Avram, Palmowski and Pistorius (Ann. Appl. Probab. 17 (2007)) solved the bail-out problem when the surplus is a general spectrally negative Lévy process, which includes the Cramér–Lundberg model, Brownian motion with drift and their sums. They showed that the optimal policy is a double barrier policy for every initial capital. This mission formalizes that result, Theorem 3 of the paper.

Timeline:

  • 1957 — de Finetti introduces dividend barriers.
  • 1978 — Harrison and Taylor: optimal control of a Brownian storage system with two reflecting barriers.
  • 1995 — Jeanblanc and Shiryaev: optimal dividends for Brownian motion with drift.
  • 2004 — Avram, Kyprianou and Pistorius: exit problems for spectrally negative Lévy processes reflected at their infimum and supremum, in terms of scale functions.
  • 2007 — Avram, Palmowski and Pistorius: Theorems 1 and 3 of the present paper; Pistorius' pathwise construction of the doubly reflected process is used.

Setting

A spectrally negative Lévy process X={Xt}t≥0X=\{X_t\}_{t\ge0}X={Xt​}t≥0​ on a filtered probability space (Ω,F,F,P)(\Omega,\mathcal F,\mathbb F,P)(Ω,F,F,P) starts at X0=0X_0=0X0​=0, has càdlàg paths, is F\mathbb FF-adapted, and has stationary increments with Xt−XsX_t-X_sXt​−Xs​ independent of Fs\mathcal F_sFs​. Its jumps are all negative, and E[eθXt]=etψ(θ)E[e^{\theta X_t}]=e^{t\psi(\theta)}E[eθXt​]=etψ(θ) for θ≥0\theta\ge0θ≥0, where the Laplace exponent is

ψ(θ)=cθ+σ22θ2+∫(−∞,0)(eθy−1−θy1{∣y∣<1}) ν(dy).\psi(\theta)=c\theta+\tfrac{\sigma^2}{2}\theta^2+\int_{(-\infty,0)}(e^{\theta y}-1-\theta y\mathbf 1_{\{|y|<1\}})\,\nu(dy).ψ(θ)=cθ+2σ2​θ2+∫(−∞,0)​(eθy−1−θy1{∣y∣<1}​)ν(dy).

With initial capital x≥0x\ge0x≥0 the surplus is x+Xx+Xx+X. The standing assumptions are: XXX does not have monotone paths; E[X1]>−∞E[X_1]>-\inftyE[X1​]>−∞ (so ψ′(0+)=E[X1]\psi'(0+)=E[X_1]ψ′(0+)=E[X1​] is finite); σ>0\sigma>0σ>0, or ∫(−1,0)∣y∣ν(dy)=∞\int_{(-1,0)}|y|\nu(dy)=\infty∫(−1,0)​∣y∣ν(dy)=∞, or ν\nuν has a density (condition (3.3)).

A policy πˉ=(L,R)\bar\pi=(L,R)πˉ=(L,R) consists of nondecreasing adapted processes with L0=R0=0L_0=R_0=0L0​=R0​=0: cumulative dividends LLL (left-continuous) and cumulative injected capital RRR (right-continuous). The controlled surplus is Vt=x+Xt−Lt+RtV_t=x+X_t-L_t+R_tVt​=x+Xt​−Lt​+Rt​. The policy is admissible if Vt≥0V_t\ge0Vt​≥0 for t>0t>0t>0 and ∫0∞e−qtdRt<∞\int_0^\infty e^{-qt}dR_t<\infty∫0∞​e−qtdRt​<∞ almost surely. Its value and the value function are

vˉπˉ(x)=E[∫0∞e−qtdLt−φ∫0∞e−qtdRt],vˉ∗(x)=sup⁡πˉ admissiblevˉπˉ(x),\bar v_{\bar\pi}(x)=E\Bigl[\int_0^\infty e^{-qt}dL_t-\varphi\int_0^\infty e^{-qt}dR_t\Bigr],\qquad \bar v_*(x)=\sup_{\bar\pi\ \text{admissible}}\bar v_{\bar\pi}(x),vˉπˉ​(x)=E[∫0∞​e−qtdLt​−φ∫0∞​e−qtdRt​],vˉ∗​(x)=πˉ admissiblesup​vˉπˉ​(x),

with discount rate q>0q>0q>0 and cost φ>1\varphi>1φ>1. The double-barrier strategy πˉ0,a\bar\pi_{0,a}πˉ0,a​ pays out (x−a)+(x-a)^+(x−a)+ at once and then the minimal dividends and injections that keep VVV in [0,a][0,a][0,a]: dLdLdL is carried by {V=a}\{V=a\}{V=a} and dRdRdR by {V=0}\{V=0\}{V=0}.

The qqq-scale function W=W(q)W=W^{(q)}W=W(q) vanishes on (−∞,0)(-\infty,0)(−∞,0), is continuous and nondecreasing on [0,∞)[0,\infty)[0,∞), and satisfies ∫0∞e−θyW(y)dy=1/(ψ(θ)−q)\int_0^\infty e^{-\theta y}W(y)dy=1/(\psi(\theta)-q)∫0∞​e−θyW(y)dy=1/(ψ(θ)−q) for θ>Φ(q)\theta>\Phi(q)θ>Φ(q), the largest root of ψ=q\psi=qψ=q. Set W‾(y)=∫0yW\overline W(y)=\int_0^yWW(y)=∫0y​W, Z=1+qW‾Z=1+q\overline WZ=1+qW, Z‾(y)=∫0yZ\overline Z(y)=\int_0^yZZ(y)=∫0y​Z. The candidate value of πˉ0,a\bar\pi_{0,a}πˉ0,a​ is

vˉa(x)=φ(Z‾(x)+ψ′(0+)/q)+Z(x)1−φZ(a)qW(a)(0≤x≤a),vˉa(x)=x−a+vˉa(a)(x>a),\bar v_a(x)=\varphi\bigl(\overline Z(x)+\psi'(0+)/q\bigr)+Z(x)\frac{1-\varphi Z(a)}{qW(a)}\quad(0\le x\le a),\qquad \bar v_a(x)=x-a+\bar v_a(a)\quad(x>a),vˉa​(x)=φ(Z(x)+ψ′(0+)/q)+Z(x)qW(a)1−φZ(a)​(0≤x≤a),vˉa​(x)=x−a+vˉa​(a)(x>a),

and the barrier level is d∗=inf⁡{a>0:[φZ(a)−1]W′(a)−φqW(a)2≤0}d^*=\inf\{a>0:[\varphi Z(a)-1]W'(a)-\varphi qW(a)^2\le0\}d∗=inf{a>0:[φZ(a)−1]W′(a)−φqW(a)2≤0}, with inf⁡∅=∞\inf\emptyset=\inftyinf∅=∞.

Formalization targets

Goal: Theorem 3

d∗<∞,vˉ∗(x)=vˉd∗(x)  (x≥0),πˉ0,d∗ exists, is admissible and attains vˉ∗(x).d^*<\infty,\qquad \bar v_*(x)=\bar v_{d^*}(x)\ \ (x\ge0),\qquad \bar\pi_{0,d^*}\ \text{exists, is admissible and attains }\bar v_*(x).d∗<∞,vˉ∗​(x)=vˉd∗​(x)  (x≥0),πˉ0,d∗​ exists, is admissible and attains vˉ∗​(x).

Milestones

  • Lemma 1: W‾(y)/W‾(a)≤W(y)/W(a)\overline W(y)/\overline W(a)\le W(y)/W(a)W(y)/W(a)≤W(y)/W(a) for 0≤y≤a0\le y\le a0≤y≤a.
  • Proposition 2, (3.17): the expected discounted undershoot Ex[e−qT0−XT0−]E_x[e^{-qT_0^-}X_{T_0^-}]Ex​[e−qT0−​XT0−​​] in closed form.
  • Theorem 1: the expected discounted dividends and injections of πˉ0,a\bar\pi_{0,a}πˉ0,a​, a>0a>0a>0, in closed form; hence vˉπˉ0,a=vˉa\bar v_{\bar\pi_{0,a}}=\bar v_avˉπˉ0,a​​=vˉa​.
  • Lemma 2(ii): d∗=0d^*=0d∗=0 if and only if σ=0\sigma=0σ=0 and ν(−∞,0)≤q/(φ−1)\nu(-\infty,0)\le q/(\varphi-1)ν(−∞,0)≤q/(φ−1).
  • Proposition 3(ii): d∗<∞d^*<\inftyd∗<∞ and vˉa≤vˉd∗\bar v_a\le\bar v_{d^*}vˉa​≤vˉd∗​ for all levels aaa.
  • Lemma 3(ii)–(iv): 1≤vˉd∗′≤φ1\le\bar v_{d^*}'\le\varphi1≤vˉd∗′​≤φ with boundary slopes; a↦vˉa(x)a\mapsto\bar v_a(x)a↦vˉa​(x) nonincreasing for a>d∗a>d^*a>d∗; vˉd∗\bar v_{d^*}vˉd∗​ concave.
  • Lemma 5: (Γvˉd∗−qvˉd∗)≤0(\Gamma\bar v_{d^*}-q\bar v_{d^*})\le0(Γvˉd∗​−qvˉd∗​)≤0 on (0,∞)(0,\infty)(0,∞), with equality on (0,d∗)(0,d^*)(0,d∗).
  • Proposition 4(ii): any C2C^2C2 solution of the variational inequality (5.9) dominates vˉ∗\bar v_*vˉ∗​.

Significance

Theorem 3 answers the bail-out problem completely: for every initial capital and every spectrally negative Lévy surplus, the optimal policy is a double barrier with an explicit level and an explicit value in terms of scale functions. In the classical problem without injections (Theorem 2 of the same paper) the analogous conclusion needs an extra generator condition, and Azcue and Muler exhibited Cramér–Lundberg models where barrier policies are not optimal. The result is the basis of later work on dividends with capital injection, transaction costs and Parisian ruin.

The result has a published proof. What this mission adds is a machine-checked version. As far as is known, no part of the theory used — Lévy processes, scale functions, doubly reflected processes, the generator of a Lévy process, singular stochastic control — has been formalized in Lean's Mathlib.

Difficulty

The value function is a supremum over all adapted singular controls. Its upper bound requires a verification argument: Itô's formula for e−qtw(Vt)e^{-qt}w(V_t)e−qtw(Vt​) with a semimartingale VVV that has jumps, a controlled bounded-variation part and a possibly nonzero continuous martingale part, followed by localization and limits. A function that satisfies the variational inequality only in the viscosity sense is not enough, so the candidate vˉd∗\bar v_{d^*}vˉd∗​ must be shown to be regular enough and to satisfy Γvˉd∗−qvˉd∗≤0\Gamma\bar v_{d^*}-q\bar v_{d^*}\le0Γvˉd∗​−qvˉd∗​≤0 everywhere on (0,∞)(0,\infty)(0,∞). Above the barrier this inequality does not follow from a martingale property: the paper derives it from concavity and a comparison with higher barriers, through the resolvent of the doubly reflected process. The lower bound requires the existence and value of the doubly reflected process, and computing that value needs fluctuation identities (two-sided exit, overshoot) that are themselves theorems about scale functions.

Formalization scope

Time is indexed by R≥0\mathbb R_{\ge0}R≥0​. The process is a structure carrying the triplet (c,σ,ν)(c,\sigma,\nu)(c,σ,ν), the paths, adaptedness, independence of future increments from Fs\mathcal F_sFs​, stationarity, the Laplace-exponent identity, and the usual conditions on the filtration. PxP_xPx​ is the law of x+Xx+Xx+X. Policy values are computed as E[∫e−qtdL]−φE[∫e−qtdR]E[\int e^{-qt}dL]-\varphi E[\int e^{-qt}dR]E[∫e−qtdL]−φE[∫e−qtdR] in the extended reals from two [0,∞][0,\infty][0,∞]-valued expectations, and the value function is a supremum in the extended reals; a policy with both expectations infinite gets −∞-\infty−∞. Stieltjes integrals include the jump at time 000. The scale function is a hypothesis IsScaleFunction (it is unique), and d∗d^*d∗ lives in [0,∞][0,\infty][0,∞], so an empty defining set gives ∞\infty∞. The double-barrier strategy is characterized by the two-sided Skorokhod conditions plus mutual singularity of dLdLdL and dRdRdR, which pins the level-000 policy of bounded-variation processes. These conditions hold almost surely, and they are stated on the post-decision surplus Vt+=x+Xt−Lt++RtV_{t+}=x+X_t-L_{t+}+R_tVt+​=x+Xt​−Lt+​+Rt​: dLdLdL is carried by {Vt+=a}\{V_{t+}=a\}{Vt+​=a} and dRdRdR by {Vt+=0}\{V_{t+}=0\}{Vt+​=0}. This is the paper's "minimal amount" (p. 4) and matches its construction on pp. 10–11. The closure-of-support wording of (4.2) on its own would also admit non-minimal lump dividends. Admissibility, Vt≥0V_t\ge0Vt​≥0 for t>0t>0t>0 together with (2.3), is likewise required almost surely.

Printed slips corrected (milestone texts are verbatim):

  • Theorem 3 and Proposition 3(ii) print "ψ′(0+)<∞\psi'(0+)<\inftyψ′(0+)<∞", which always holds; the intended ψ′(0+)>−∞\psi'(0+)>-\inftyψ′(0+)>−∞ is used.
  • (3.3) prints ∫−10x ν(dx)=∞\int_{-1}^0x\,\nu(dx)=\infty∫−10​xν(dx)=∞ for ∫(−1,0)∣x∣ ν(dx)=∞\int_{(-1,0)}|x|\,\nu(dx)=\infty∫(−1,0)​∣x∣ν(dx)=∞.
  • (3.4) prints e−θxe^{-\theta x}e−θx in a dydydy-integral.
  • p. 20 prints the extension vˉd∗(x)+φx\bar v_{d^*}(x)+\varphi xvˉd∗​(x)+φx for vˉd∗(0)+φx\bar v_{d^*}(0)+\varphi xvˉd∗​(0)+φx.
  • Lemma 3(iv) is printed for every a>0a>0a>0 but proved and used only for a=d∗a=d^*a=d∗, and is stated for a=d∗a=d^*a=d∗.
  • Proposition 3(ii) at a=0a=0a=0 is stated only for bounded variation, where vˉ0\bar v_0vˉ0​ is defined.

The goal cannot be satisfied trivially. It asserts equality of the value function with vˉd∗\bar v_{d^*}vˉd∗​, not only an inequality. Existence of the optimal policy is part of the conclusion. Generator statements carry integrability of the jump integrand, so a non-integrable integrand cannot make them true with the junk value 000.

A complete development needs Lévy processes and their Laplace exponents, scale functions, first-passage and two-sided exit identities, reflected and doubly reflected processes, Itô's formula for semimartingales with jumps, and a verification theorem for singular control. All of these are reusable well beyond this mission. Contributions of any of these foundations are welcome, as are proofs of the analytic milestones (Lemma 3, Proposition 3(ii)) from the scale-function properties.

Selected references

  • F. Avram, Z. Palmowski, M. R. Pistorius, On the optimal dividend problem for a spectrally negative Lévy process, Ann. Appl. Probab. 17 (2007) 156–180. https://arxiv.org/abs/math/0702893
  • F. Avram, A. E. Kyprianou, M. R. Pistorius, Exit problems for spectrally negative Lévy processes and applications to (Canadized) Russian options, Ann. Appl. Probab. 14 (2004) 215–238. https://doi.org/10.1214/aoap/1075828052
  • M. R. Pistorius, On doubly reflected completely asymmetric Lévy processes, Stochastic Process. Appl. 107 (2003) 131–143. https://doi.org/10.1016/S0304-4149(03)00065-9
  • J. M. Harrison, A. J. Taylor, Optimal control of a Brownian storage system, Stochastic Process. Appl. 6 (1978) 179–194. https://doi.org/10.1016/0304-4149(78)90059-5
  • A. E. Kyprianou, Introductory Lectures on Fluctuations of Lévy Processes with Applications, Springer, 2006. https://doi.org/10.1007/978-3-540-31343-4
15 thms1 active userReviewed
Algorithmic Game TheoryOperations ResearchOptimization·Captain: mikedeng1

A Supply Chain Theory of Factoring and Reverse Factoring 1: The Supplier's Equilibrium Choice Among Recourse, Non-Recourse and Reverse FactoringResearch Paper

Motivation

Small and medium-sized suppliers that sell to large retailers typically ship first and are paid weeks or months later. Until the invoice is paid, the supplier's cash is trapped in accounts receivable, while production has to be financed up front. Factoring (selling or pledging the receivable to a financial intermediary for immediate cash) and reverse factoring (a buyer-initiated program in which the retailer approves the invoice and the supplier is paid early by the retailer's bank) are the standard remedies, and reverse factoring programs of large buyers are often coupled with an extension of payment terms. Which of these instruments a supplier should use, and how the answer depends on the credit ratings of the two firms, is the question addressed by Kouvelis and Xu, "A Supply Chain Theory of Factoring and Reverse Factoring", Management Science 67(10):6071–6088, 2021 (doi:10.1287/mnsc.2020.3788). The paper embeds each financing scheme in the pull wholesale-price game of Cachon (2004) and compares the resulting equilibria.

Setting

A retailer (the Stackelberg leader) offers a wholesale price www to a capital-constrained supplier (the follower), who then chooses a production quantity q≥0q \ge 0q≥0 and carries the inventory; the retail price ppp is fixed, and 0<c<p0 < c < p0<c<p is the unit production cost. Demand D≥0D \ge 0D≥0 has density fff, complementary distribution function Fˉ\bar FFˉ, finite mean, f>0f > 0f>0 on [0,Z][0, \mathbb Z][0,Z] (Z≤+∞\mathbb Z \le +\inftyZ≤+∞), and a strictly increasing failure rate z=f/Fˉz = f/\bar Fz=f/Fˉ (strict IFR). Expected sales are S(q)=∫0qFˉ(ξ) dξS(q) = \int_0^q \bar F(\xi)\,d\xiS(q)=∫0q​Fˉ(ξ)dξ, and k(q)=S(q)/Fˉ(q)k(q) = S(q)/\bar F(q)k(q)=S(q)/Fˉ(q).

Each firm j∈{s,r}j \in \{s, r\}j∈{s,r} has a credit rating Cj∈(Cmin⁡,Cmax⁡)C_j \in (C_{\min}, C_{\max})Cj​∈(Cmin​,Cmax​). The default probability ρj=ρ(Cj)∈[0,1]\rho_j = \rho(C_j) \in [0,1]ρj​=ρ(Cj​)∈[0,1] is strictly decreasing in the rating, and the interest-rate premium ηj=η(Cj)>0\eta_j = \eta(C_j) > 0ηj​=η(Cj​)>0 is decreasing. Production takes a lead time t1t_1t1​, payment follows after a term t2t_2t2​, and the supplier faces a Poisson liquidity shock of rate λs\lambda_sλs​ during production (the retailer, rate λr\lambda_rλr​). Under a scheme i∈{F,N,R}i \in \{\mathcal F, \mathcal N, \mathcal R\}i∈{F,N,R} (recourse, non-recourse, reverse factoring, the last with a payment extension τ≥0\tau \ge 0τ≥0), the supplier's expected profit is

πi(q;w)=(1−ρs)(Λie−λst1wS(q)−cqeηst1),\pi_i(q; w) = (1-\rho_s)\big(\Lambda_i e^{-\lambda_s t_1} w S(q) - c q e^{\eta_s t_1}\big),πi​(q;w)=(1−ρs​)(Λi​e−λs​t1​wS(q)−cqeηs​t1​),

with coefficients ΛF=(1−ρr)+(1−ρs)−eηst2\Lambda_{\mathcal F} = (1-\rho_r) + (1-\rho_s) - e^{\eta_s t_2}ΛF​=(1−ρr​)+(1−ρs​)−eηs​t2​, ΛN=e−ηrt2(1−ρr)\Lambda_{\mathcal N} = e^{-\eta_r t_2}(1-\rho_r)ΛN​=e−ηr​t2​(1−ρr​), ΛR=e−ηr(t2+τ)\Lambda_{\mathcal R} = e^{-\eta_r(t_2+\tau)}ΛR​=e−ηr​(t2​+τ), and effective unit cost ci=c e(ηs+λs)t1/Λic_i = c\,e^{(\eta_s+\lambda_s)t_1}/\Lambda_ici​=ce(ηs​+λs​)t1​/Λi​. The retailer's profit is e−λst1(1−ρr)(p−w)S(q)e^{-\lambda_s t_1}(1-\rho_r)(p-w)S(q)e−λs​t1​(1−ρr​)(p−w)S(q) under factoring and carries the extra factor 2−e−λrτ2 - e^{-\lambda_r \tau}2−e−λr​τ under reverse factoring.

Formalization targets

Goal: Proposition 5

For a given τ≥0\tau \ge 0τ≥0, let C3\mathbb C_3C3​ be the unique supplier rating with ΛF=ΛR\Lambda_{\mathcal F} = \Lambda_{\mathcal R}ΛF​=ΛR​, C1\mathbb C_1C1​ the one with ΛF=ΛN\Lambda_{\mathcal F} = \Lambda_{\mathcal N}ΛF​=ΛN​, and CF,CN,CR\mathbb C_{\mathcal F}, \mathbb C_{\mathcal N}, \mathbb C_{\mathcal R}CF​,CN​,CR​ the feasibility thresholds (ci=pc_i = pci​=p). With all three schemes available,

e−ηrτ<1−ρr:F adopted  ⟺  Cs>CF∨C1,N adopted  ⟺  CN<Cs≤C1;e^{-\eta_r\tau} < 1-\rho_r:\quad \mathcal F \text{ adopted} \iff C_s > \mathbb C_{\mathcal F}\vee\mathbb C_1,\qquad \mathcal N \text{ adopted} \iff \mathbb C_{\mathcal N} < C_s \le \mathbb C_1;e−ηr​τ<1−ρr​:F adopted⟺Cs​>CF​∨C1​,N adopted⟺CN​<Cs​≤C1​; 1−ρr≤e−ηrτ:F adopted  ⟺  Cs>CF∨C3,R adopted  ⟺  CR<Cs≤C3.1-\rho_r \le e^{-\eta_r\tau}:\quad \mathcal F \text{ adopted} \iff C_s > \mathbb C_{\mathcal F}\vee\mathbb C_3,\qquad \mathcal R \text{ adopted} \iff \mathbb C_{\mathcal R} < C_s \le \mathbb C_3.1−ρr​≤e−ηr​τ:F adopted⟺Cs​>CF​∨C3​,R adopted⟺CR​<Cs​≤C3​.

Milestones

  1. Proposition 2 (p. 6078): recourse factoring is feasible iff Cs>CFC_s > \mathbb C_{\mathcal F}Cs​>CF​, and then the unique equilibrium solves pFˉ(q)=cF[1+z(q)k(q)]p\bar F(q) = c_{\mathcal F}[1+z(q)k(q)]pFˉ(q)=cF​[1+z(q)k(q)], w=cF/Fˉ(q)w = c_{\mathcal F}/\bar F(q)w=cF​/Fˉ(q).
  2. Proposition 3 (p. 6079): the same for non-recourse factoring with cNc_{\mathcal N}cN​.
  3. §5.2 (p. 6082, Proposition EC.2): the same for reverse factoring with cR(τ)c_{\mathcal R}(\tau)cR​(τ).
  4. §4.4 (p. 6080, Lemma EC.2): between feasible schemes, the supplier's equilibrium profits are ordered as the Λi\Lambda_iΛi​.
  5. Proposition 4 (p. 6080): the choice between F\mathcal FF and N\mathcal NN alone.

A follow-on item states Corollary 1(i) (p. 6081): when the retailer's rating is at least the supplier's, non-recourse factoring strictly dominates recourse factoring whenever the latter is feasible.

Significance

Proposition 5 is the paper's main prediction: non-recourse factoring is the choice of medium-rated suppliers, recourse factoring of highly rated ones, and reverse factoring with a fixed payment extension is chosen only by low-to-medium-rated suppliers and only if the retailer's rating is not too high relative to the extension. The paper uses it to explain the documented reluctance of suppliers to join reverse-factoring programs that come with extended payment terms (Wuttke, Rosenzweig and Heese 2019; Corsten 2010, a teaching case), and it is the starting point of the paper's Proposition 6 on the retailer's optimal payment extension, which is the subject of the second mission of this series.

The results are proved in the paper's Online Appendix B. To the best of available knowledge none of them, and no model of supply chain finance, has a machine-checked proof. A formal development would also supply a reusable treatment of the pull wholesale-price game under a strictly IFR demand with an arbitrary effective unit cost, of which Propositions 2, 3 and EC.2 are three instances.

Difficulty

The case analysis of Proposition 5 is short once the milestones are available; the substance lies in them. Proposition 2 requires solving a bilevel problem: the follower's best response has to be identified for every wholesale price, including prices at which the supplier produces nothing, and the leader's objective has to be shown to have a single maximizer. The natural route through the first-order condition needs the IFR property to exclude multiple stationary points, and it has to cope with a support end Z\mathbb ZZ that may be finite (where Fˉ\bar FFˉ vanishes and kkk, zzz lose meaning) or infinite. Lemma EC.2 requires the equilibrium supplier profit as an explicit function of the effective cost and a monotonicity argument for it; comparing the Λi\Lambda_iΛi​ directly, without the equilibrium, does not prove it. The proofs themselves are not in the main text.

Formalization scope

Demand is a probability measure μ on ℝ with a density f, the support end Z : EReal, and the complementary distribution function Fbar μ x = μ((x, ∞)). The model data, the coefficients coef, the effective costs cF, cN, cR, the two profit functions, best responses, equilibria, feasibility and adoption are definitions; all theorems are for arbitrary data satisfying Params.Valid and DemandModel. Readings fixed by the formalization:

  • "Feasible": some wholesale price w≥0w \ge 0w≥0 with a supplier best response gives the retailer strictly positive profit. It is not defined as ci<pc_i < pci​<p, which is what Propositions 2 and 3 prove.
  • "Adopted" / "should be adopted": the scheme is feasible and its equilibrium supplier profit is best among the feasible available schemes, with ties resolved R≻N≻F\mathcal R \succ \mathcal N \succ \mathcal FR≻N≻F, the order forced by the paper's boundaries (Cs=C1C_s = \mathbb C_1Cs​=C1​, Cs=C3C_s = \mathbb C_3Cs​=C3​, Cr=C2C_r = \mathbb C_2Cr​=C2​). Profits are compared, not the Λi\Lambda_iΛi​.
  • "The unique value of CsC_sCs​ that satisfies …": a point of (Cmin⁡,Cmax⁡)(C_{\min}, C_{\max})(Cmin​,Cmax​) satisfying the equation and the only such point. The equation cF=pc_{\mathcal F} = pcF​=p is cross-multiplied, c e(ηs+λs)t1=pΛFc\,e^{(\eta_s+\lambda_s)t_1} = p\Lambda_{\mathcal F}ce(ηs​+λs​)t1​=pΛF​, since ΛF\Lambda_{\mathcal F}ΛF​ may be ≤0\le 0≤0.
  • "Cr>C2C_r > \mathbb C_2Cr​>C2​" is read through the paper's equivalence τ>−ηr−1ln⁡(1−ρr)\tau > -\eta_r^{-1}\ln(1-\rho_r)τ>−ηr−1​ln(1−ρr​), i.e. e−ηrτ<1−ρre^{-\eta_r\tau} < 1-\rho_re−ηr​τ<1−ρr​: the stated assumptions give no single crossing in CrC_rCr​.
  • "The unique equilibrium can be derived from …": the equilibrium exists and is unique, and it is exactly the solution of the system with q∈(0,Z)q \in (0, \mathbb Z)q∈(0,Z); this part is stated under feasibility.
  • "Increasing/decreasing" is weak unless stated (p. 6075); ρ\rhoρ is strictly decreasing, η\etaη weakly.
  • Continuity of fff is read on [0,Z][0,\mathbb Z][0,Z], and Z\mathbb ZZ as the upper end of the support.
  • Additions: wholesale prices are nonnegative; the retailer's reverse-factoring profit is the last line of the p. 6082 display (the middle line has a stray factor www).
  • Corollary 1(i): "higher credit rating" is read as Cr≥CsC_r \ge C_sCr​≥Cs​, "dominates" as a strictly larger equilibrium supplier profit whenever recourse is feasible.

A formalization in which feasibility or adoption is defined by the inequalities being proved (ci<pc_i < pci​<p, largest Λi\Lambda_iΛi​) makes the propositions trivial and is ruled out by the definitions above. The profit functions (7), (10) and the p. 6082 display are taken as the model; the derivation from bank and factor pricing (Eqs. (1), (5), (6), Lemma 1) is not formalized, and the unstated assumptions of Table EC.2 are not included. Contributions of reusable lemmas on SSS, kkk and zzz under strict IFR are welcome.

Selected references

  • P. Kouvelis and F. Xu, A Supply Chain Theory of Factoring and Reverse Factoring, Management Science 67(10):6071–6088, 2021. https://doi.org/10.1287/mnsc.2020.3788
  • G. P. Cachon, The Allocation of Inventory Risk in a Supply Chain: Push, Pull, and Advance-Purchase Discount Contracts, Management Science 50(2):222–238, 2004. https://doi.org/10.1287/mnsc.1030.0189
  • D. A. Wuttke, E. Rosenzweig, H. S. Heese, An Empirical Analysis of Supply Chain Finance Adoption, Journal of Operations Management 65(3):242–261, 2019.
  • P. Kouvelis and W. Zhao, Who Should Finance the Supply Chain? Impact of Credit Ratings on Supply Chain Decisions, Manufacturing & Service Operations Management 20(1):19–35, 2018.
8 thms1 active userReviewed
🏆Completed
Linear algebraMachine Learning·Captain: Minghui

Fine-Tuning Can Distort Pretrained Features: Perfect-Feature LP-FT SeparationResearch Paper

Why initialization matters for transfer learning

Transfer learning starts with a representation learned on an earlier task and adapts it to a new one. Two common choices are linear probing, which changes only the final linear predictor, and fine-tuning, which changes the representation as well. These procedures optimize related training objectives, but their behavior away from the training data can differ. Kumar and coauthors study this distinction through two-layer linear networks, alongside experiments with nonlinear networks. This mission formalizes their perfect-feature LP-FT result, rather than the empirical claims or the general imperfect-feature comparison. See Section 3.4, Proposition 3.7, PDF p. 10.

LP-FT first learns a head by linear probing and then uses that head to initialize full fine-tuning. The perfect-feature setting isolates the effect of head initialization: the representation already contains exactly the features needed to predict the labels, but the head initially need not use them correctly. The mathematical question is whether joint training preserves or loses the representation's ability to predict outside the observed training subspace.

Linear predictors, training data, and OOD loss

An input is a vector x∈Rdx\in\mathbb R^dx∈Rd. A feature extractor is a matrix B∈Rk×dB\in\mathbb R^{k\times d}B∈Rk×d, and a head is a vector v∈Rkv\in\mathbb R^kv∈Rk. Together they predict v⊤Bxv^\top Bxv⊤Bx, with effective weight vector B⊤vB^\top vB⊤v. The fixed matrix X∈Rn×dX\in\mathbb R^{n\times d}X∈Rn×d contains the nnn training inputs as rows. Their span is S=rowspace⁡(X)S=\operatorname{rowspace}(X)S=rowspace(X), with dimension mmm.

The ground truth has orthonormal-row features B⋆B_\starB⋆​ and a nonzero head v⋆v_\starv⋆​. Write w⋆=B⋆⊤v⋆w_\star=B_\star^\top v_\starw⋆​=B⋆⊤​v⋆​ and Y=Xw⋆Y=Xw_\starY=Xw⋆​. Perfect pretrained features mean B0=UB⋆B_0=UB_\starB0​=UB⋆​ for an orthogonal matrix UUU. The corresponding aligned head is u=Uv⋆u=Uv_\staru=Uv⋆​. The dimensions satisfy 1≤k≤m1\le k\le m1≤k≤m and m+k<dm+k<dm+k<d.

The two geometric assumptions require the orthogonal projections from R0=rowspace⁡(B0)R_0=\operatorname{rowspace}(B_0)R0​=rowspace(B0​) into SSS and into S⊥S^\perpS⊥ to be injective. In this dimension regime these are exactly the positive largest-principal-angle cosine conditions used by the paper. They demand more than two subspaces having some nonorthogonal directions. The Lean definition spells out injectivity of v↦ΠTB0⊤vv\mapsto\Pi_T B_0^\top vv↦ΠT​B0⊤​v for each T∈{S,S⊥}T\in\{S,S^\perp\}T∈{S,S⊥}. See Definition 3.2 and Appendix A.1, PDF pp. 7 and 22-23.

An out-of-distribution law μ\muμ is any probability measure on Rd\mathbb R^dRd with a finite second moment and positive-definite uncentered second-moment matrix Σ=Eμ[xx⊤]\Sigma=\mathbb E_\mu[xx^\top]Σ=Eμ​[xx⊤]. Its mean need not be zero. Define

LOOD(v,B)=Ex∼μ[(v⊤Bx−w⋆⊤x)2].L_{\rm OOD}(v,B)=\mathbb E_{x\sim\mu} [(v^\top Bx-w_\star^\top x)^2].LOOD​(v,B)=Ex∼μ​[(v⊤Bx−w⋆⊤​x)2].

Both training methods use the unnormalized loss L^(v,B)=∥XB⊤v−Y∥22\widehat L(v,B)=\|XB^\top v-Y\|_2^2L(v,B)=∥XB⊤v−Y∥22​. Fine-tuning follows its gradient flow in both parameters; linear probing keeps B=B0B=B_0B=B0​. Time is real and nonnegative. These are the paper's equations (3.2)-(3.3), PDF p. 6.

Formalization targets

The goal is Proposition 3.7 in an explicit nonzero-signal regime. For σ>0\sigma>0σ>0, initialize an FT head with independent Gaussian coordinates, v0∼N(0,σ2Ik)v_0\sim\mathcal N(0,\sigma^2I_k)v0​∼N(0,σ2Ik​). Establish

Pr⁡ ⁣[∀t≥0,LOOD(vFT(t),BFT(t))>0]=1.\Pr\!\left[\forall t\ge0,\quad L_{\rm OOD}(v_{\rm FT}(t),B_{\rm FT}(t))>0\right]=1.Pr[∀t≥0,LOOD​(vFT​(t),BFT​(t))>0]=1.

Linear probing, from any initial head, must converge to uuu. Fine-tuning initialized at its limit must satisfy

vLP(t)⟶u,∀t≥0,LOOD(vLP-FT(t),BLP-FT(t))=0.v_{\rm LP}(t)\longrightarrow u,\qquad \forall t\ge0,\quad L_{\rm OOD}(v_{\rm LP\text{-}FT}(t),B_{\rm LP\text{-}FT}(t))=0.vLP​(t)⟶u,∀t≥0,LOOD​(vLP-FT​(t),BLP-FT​(t))=0.

The probability-one event applies to all times simultaneously. The goal also asserts existence of the relevant global flows; a conditional claim about a possibly nonexistent trajectory would not suffice. The statement does not assert a numerical error lower bound or a positive time-infimum.

Seven milestones supply the supporting results: global flow existence and FT uniqueness; unchanged features orthogonal to the training span; the balancedness invariant; the second-moment identity for OOD risk; almost-sure Gaussian head misalignment; exact LP recovery; and stationarity after LP initialization. The principal source is Appendices A.2 and A.7, PDF pp. 23-31 and 45-47.

What the result establishes

The result distinguishes two initializations of the same joint-training procedure. In this idealized setting, a head obtained by linear probing gives zero OOD loss throughout subsequent fine-tuning, while a Gaussian head almost surely has positive OOD loss at every finite time. The conclusion concerns population squared prediction error, not classification accuracy or a finite test-set estimate.

The paper establishes the mathematical claim; this mission asks for a Lean proof of the stated model and result. The scope is deliberately limited to perfect pretrained features. It does not claim an LP-FT upper bound for imperfect features, which the authors identify as a further challenge in Section 3.4, PDF p. 10. A completed development would also provide reusable components for finite dimensional gradient flows, factorized linear models, and population risk.

Why the proof needs the training dynamics

The training loss alone does not select a unique effective predictor in an overparameterized problem. Knowing that a predictor fits the observed examples therefore does not determine its OOD loss. Formalization must track the head and feature extractor together, and it must distinguish parameter stationarity from a claim that a derivative happens to vanish at one time. The Gaussian conclusion also requires one event controlling an uncountable set of times; separate probability-one statements for individual times would be weaker.

Formalization scope and conventions

Vectors use Mathlib's finite dimensional real Euclidean spaces. Matrices are represented as continuous linear maps, with Euclidean adjoints and operator norms. The feature update is written explicitly as the Frobenius-gradient equation; it is not a gradient with respect to the operator norm. Differentiability is imposed within [0,∞)[0,\infty)[0,∞), including the right derivative at zero.

The dimensions, nonzero target, positive Gaussian scale, finite second moments, and projection injectivity are explicit. The nonzero target restricts the formalization to the regime of the Gaussian alignment argument in Lemma A.12; k≤mk\le mk≤m makes the identifiability condition used in Proposition A.20 precise. The random-head law is the scaled standard Gaussian measure. No randomness of the fixed training matrix or independence from an additional data draw is assumed.

The model contains no assumed convergence, invariant, or desired risk bound. Each of those is a theorem obligation. The well-posedness milestone makes explicit an analytic prerequisite of the source's flow notation. The risk milestone uses the identity in (A.29)-(A.32), avoiding the reversed inequality printed in (A.28). The quantitative constant in Theorem 3.3 is outside this mission. Source-aligned proofs and the supporting analysis infrastructure are welcome; changing the learning rule or assuming a milestone inside the model would change the task.

Selected references

  • Ananya Kumar, Aditi Raghunathan, Robbie Jones, Tengyu Ma, and Percy Liang, Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution, ICLR 2022, arXiv:2202.10054v1. Main target: Section 3.4, Proposition 3.7, PDF p. 10, equations (3.10)-(3.11); proof: Appendix A.7, PDF pp. 45-47, Proposition A.20 and (A.208)-(A.218). Supporting invariants: Appendix A.2, PDF p. 24, Lemmas A.3-A.4, equations (A.15)-(A.20). Gaussian alignment: Appendix A.3, PDF pp. 34-35, Lemmas A.11-A.12.
9 thms1 active userReviewed
Dynamical SystemsNumber Theory·Captain: mysticflounder

Tao 2022: Almost All Collatz Orbits Attain Almost Bounded ValuesResearch Paper

Motivation

The Collatz map sends an even positive integer nnn to n/2n/2n/2 and an odd one to 3n+13n+13n+1. The Collatz conjecture asserts that every orbit of this map eventually reaches 111. It remains an open problem.

Because the conjecture is open, a large part of the literature proves weaker statements that hold for almost all starting values. These results bound how small an orbit gets, not whether it reaches 111. They are the strongest unconditional evidence for the conjecture. Terence Tao's 2022 theorem is the strongest result of this type: outside a set of logarithmic density zero, every orbit drops below any prescribed function that tends to infinity, however slowly (Tao 2022).

Timeline

Attributions follow the survey in Tao 2022, Section 1. Here Colmin⁡(N)\mathrm{Col}_{\min}(N)Colmin​(N) is the least value in the orbit of NNN.

  • 1976 to 1979, Terras and Everett. Colmin⁡(N)<N\mathrm{Col}_{\min}(N) < NColmin​(N)<N for almost all NNN, in natural density.
  • 1979, Allouche. Colmin⁡(N)<Nθ\mathrm{Col}_{\min}(N) < N^{\theta}Colmin​(N)<Nθ for almost all NNN, for every θ>0.869\theta > 0.869θ>0.869.
  • 1994, Korec. The same bound for every θ>log⁡3/log⁡4≈0.7924\theta > \log 3/\log 4 \approx 0.7924θ>log3/log4≈0.7924.
  • 2019 preprint, 2022 publication, Tao. The power NθN^{\theta}Nθ is replaced by any f(N)→∞f(N) \to \inftyf(N)→∞, in logarithmic density instead of natural density (Forum of Mathematics, Pi 10, e12).
  • 2026, ProofAtlas. A complete machine-checked proof of Tao's theorem in Lean 4 is published (formalization record).

Setting

Let collatzStep:N→N\mathrm{collatzStep} : \mathbb{N} \to \mathbb{N}collatzStep:N→N be

collatzStep(n)={n/2n even,3n+1n odd.\mathrm{collatzStep}(n) = \begin{cases} n/2 & n \text{ even},\\ 3n+1 & n \text{ odd}. \end{cases}collatzStep(n)={n/23n+1​n even,n odd.​

Write collatzStep[k]\mathrm{collatzStep}^{[k]}collatzStep[k] for the kkk-fold iterate, with collatzStep[0](n)=n\mathrm{collatzStep}^{[0]}(n) = ncollatzStep[0](n)=n. The orbit of nnn is the set of values collatzStep[k](n)\mathrm{collatzStep}^{[k]}(n)collatzStep[k](n) for k≥0k \ge 0k≥0. It includes nnn itself.

The Syracuse map acts on odd integers. It sends an odd nnn to the odd part of 3n+13n+13n+1, which is syracuseStep(n)=(3n+1)/2ν2(3n+1)\mathrm{syracuseStep}(n) = (3n+1)/2^{\nu_2(3n+1)}syracuseStep(n)=(3n+1)/2ν2​(3n+1). The Syracuse orbit minimum syracuseOrbitMin(n)\mathrm{syracuseOrbitMin}(n)syracuseOrbitMin(n) is the least value syracuseStep[k](n)\mathrm{syracuseStep}^{[k]}(n)syracuseStep[k](n) over k≥0k \ge 0k≥0.

For a set AAA of positive integers and a real cutoff x≥2x \ge 2x≥2, the logarithmic mass of AAA up to xxx is

weightedLogMassReal(x)=1log⁡x∑1≤n≤xn∈A1n.\mathrm{weightedLogMassReal}(x) = \frac{1}{\log x} \sum_{\substack{1 \le n \le x \\ n \in A}} \frac{1}{n}.weightedLogMassReal(x)=logx1​1≤n≤xn∈A​∑​n1​.

A set of positive integers has logarithmic density ddd when this quantity tends to ddd as x→∞x \to \inftyx→∞. Since ∑n≤x1/n=log⁡x+O(1)\sum_{n \le x} 1/n = \log x + O(1)∑n≤x​1/n=logx+O(1), all positive integers together have logarithmic density 111, and the odd positive integers have logarithmic density 1/21/21/2.

A function f:N→Rf : \mathbb{N} \to \mathbb{R}f:N→R tends to infinity on positive inputs when for every real MMM there is an NNN such that f(n)>Mf(n) > Mf(n)>M for all n≥Nn \ge Nn≥N with n>0n > 0n>0.

Formalization targets

Goal: Tao's Theorem 1.3

For every fff that tends to infinity on positive inputs,

lim⁡x→∞1log⁡x∑1≤n≤x∃k, collatzStep[k](n)<f(n)1n=1.\lim_{x \to \infty} \frac{1}{\log x} \sum_{\substack{1 \le n \le x \\ \exists k,\ \mathrm{collatzStep}^{[k]}(n) < f(n)}} \frac{1}{n} = 1.x→∞lim​logx1​1≤n≤x∃k, collatzStep[k](n)<f(n)​∑​n1​=1.

This is the Lean theorem collatz_almost_bounded_logarithmic. It fixes no rate and no constant, so later improvements do not invalidate it.

Syracuse form: Tao's Theorem 1.6

For every fff that tends to infinity on odd positive inputs, the odd nnn whose Syracuse orbit contains a value below f(n)f(n)f(n) have logarithmic density 1/21/21/2, which is all of the odd integers. This is syracuse_almost_bounded_logarithmic.

Quantitative form: Tao's Theorem 3.1

There are constants C,c>0C, c > 0C,c>0 such that for all real M≥2M \ge 2M≥2 and x≥2x \ge 2x≥2,

1log⁡x∑1≤n≤x, n oddsyracuseOrbitMin(n)>M1n≤C(log⁡M)c.\frac{1}{\log x} \sum_{\substack{1 \le n \le x,\ n \text{ odd} \\ \mathrm{syracuseOrbitMin}(n) > M}} \frac{1}{n} \le \frac{C}{(\log M)^{c}}.logx1​1≤n≤x, n oddsyracuseOrbitMin(n)>M​∑​n1​≤(logM)cC​.

This is syracuse_uniform_logarithmic_tail_bound. The three targets are listed from weakest to strongest. In Tao's paper, the goal and Theorem 1.6 both follow from Theorem 3.1.

Significance

The result. Theorem 1.3 improves the earlier almost-all bounds on Collatz orbit minima from a power of NNN to any function that tends to infinity. For example, Colmin⁡(N)<log⁡log⁡log⁡log⁡N\mathrm{Col}_{\min}(N) < \log\log\log\log NColmin​(N)<loglogloglogN for almost all NNN. The theorem does not settle the conjecture. A single divergent orbit or nontrivial cycle is not excluded, because its predecessors form a set with bounded orbit minima.

The formalization. The theorem is proved, and a complete formalization already exists. The ProofAtlas record Tao's Almost-Bounded Collatz Orbits proves the statement taoAlmostBoundedColMin_checked with the same step map and an orbit that includes its starting value. That package has 397 first-party Lean files and about 124,000 source lines. ProofAtlas records a passing build with no sorry and the axiom profile propext, Classical.choice, Quot.sound. The source is released under the Apache-2.0 licence at commit d53c8de00056fb05999e44589f2399d7efa026e4, with copyright held by Advameg, Inc. It was built with Lean v4.30.0-rc2.

The remaining work for this mission is therefore a port, not new mathematics. The existing Lean development must be moved to Lean v4.33.1 and Mathlib 0df444a, and then connected to the goal statement above.

Difficulty

The mathematical difficulty is already resolved in the existing proof. The classical first-descent argument shows that a typical orbit drops below its starting value. Repeating that argument fails, because after one descent the new value is no longer distributed like a random integer of its size, and the error compounds at each repetition. That is why earlier results stop at a power bound NθN^{\theta}Nθ.

The difficulty of this mission is engineering. The port moves a development of about 124,000 lines forward across three minor Lean releases and the corresponding Mathlib changes. Lemma renames, deprecations, and changes in tactic behaviour must be repaired file by file without changing any statement.

Formalization scope

  • Step map. collatzStep n = if Even n then n / 2 else 3 * n + 1, so collatzStep(0)=0\mathrm{collatzStep}(0) = 0collatzStep(0)=0. This is the same definition as in the ProofAtlas source.
  • Orbit. The witness kkk ranges over all natural numbers, including k=0k = 0k=0.
  • Density. The goal uses weightedLogMassReal: a real cutoff xxx, the natural indices 0≤n≤⌊x⌋0 \le n \le \lfloor x \rfloor0≤n≤⌊x⌋, weights 1/n1/n1/n, and normalization by log⁡x\log xlogx. The ProofAtlas statement instead uses integer cutoffs and normalization by the harmonic sum ∑n≤N1/n\sum_{n \le N} 1/n∑n≤N​1/n. A short bridge between the two normalizations is part of the required work.
  • Threshold. The hypothesis on fff constrains only positive inputs. The value f(0)f(0)f(0) is irrelevant.
  • Non-triviality. The case k=0k = 0k=0 alone covers only inputs with n<f(n)n < f(n)n<f(n). For slowly growing fff this set is finite, so the goal is not satisfied by the starting value.

The goal is already reduced, through accepted proofs, to the single open statement syracuse_uniform_logarithmic_tail_bound. Two routes therefore close the mission. The first is to port the ProofAtlas development and prove the goal directly through the density bridge. The second is to prove Theorem 3.1 in its stated form. Contributions welcome include ported modules as reusable solutions, the density bridge, and a proof of Theorem 3.1. Every ported file must keep the Apache-2.0 notice and attribution to the ProofAtlas source.

Selected references

  • Terence Tao, Almost all orbits of the Collatz map attain almost bounded values, Forum of Mathematics, Pi 10 (2022), e12. https://doi.org/10.1017/fmp.2022.8 and https://arxiv.org/abs/1909.03562
  • ProofAtlas, Tao's Almost-Bounded Collatz Orbits, Lean 4 formalization, source commit d53c8de00056fb05999e44589f2399d7efa026e4, Apache-2.0, 2026. https://proofatlas.ai/formalizations/tao-almost-bounded-orbits/
  • Jeffrey C. Lagarias (editor), The Ultimate Challenge: The 3x+1 Problem, American Mathematical Society, 2010. https://bookstore.ams.org/MBK/78
3 thms1 active userReviewed
Control TheoryDynamical SystemsOperations Research+1·Captain: mikedeng1

Stabilization of Hybrid Systems by Feedback Control Based on Discrete-Time State Observations I: Almost Sure Asymptotic StabilityResearch Paper

Motivation

Many engineered systems switch between a finite number of operating modes at random times: a power grid after a line failure, a networked controller whose links drop, a manufacturing plant whose machines break down and are repaired. A standard model for such systems is a hybrid stochastic differential equation, also called an SDE with Markovian switching: the state follows an Itô equation whose coefficients depend on a mode that evolves as a continuous-time Markov chain. The monograph of Mao and Yuan (Stochastic Differential Equations with Markovian Switching, 2006) develops the stability theory of these equations.

A controller that stabilizes such a system usually needs the current state. In practice the state is sampled: it is observed at times 0,τ,2τ,…0,\tau,2\tau,\dots0,τ,2τ,… and the control is held between observations. Mao (Automatica 49, 2013) showed that, under a global Lipschitz condition on the drift and diffusion, a feedback control based on discrete-time observations stabilizes a hybrid SDE in the sense of mean-square exponential stability when τ\tauτ is small enough. You, Liu, Lu, Mao and Qiu (SIAM J. Control Optim. 53(2), 2015) replaced that condition by local Lipschitz continuity plus linear growth, gave an explicit bound (3.5) on the admissible observation interval τ\tauτ, and proved H∞H_\inftyH∞​-stability, mean-square asymptotic stability, almost sure asymptotic stability and exponential stability of the controlled system. This mission formalizes the almost sure asymptotic stability result, Theorem 3.4, and the results it is built on.

Setting

Let (Ω,F,{Ft}t≥0,P)(\Omega,\mathcal F,\{\mathcal F_t\}_{t\ge0},\mathbb P)(Ω,F,{Ft​}t≥0​,P) be a probability space with a filtration satisfying the usual conditions (increasing, right-continuous, F0\mathcal F_0F0​ contains the null sets). On it live an mmm-dimensional {Ft}\{\mathcal F_t\}{Ft​}-Brownian motion www and a right-continuous {Ft}\{\mathcal F_t\}{Ft​}-Markov chain rrr on S={1,…,N}S=\{1,\dots,N\}S={1,…,N} with generator Γ=(γij)\Gamma=(\gamma_{ij})Γ=(γij​) (γij≥0\gamma_{ij}\ge0γij​≥0 for i≠ji\ne ji=j, zero row sums), independent of www. Fix τ>0\tau>0τ>0 and the sampling time δt=[t/τ]τ\delta_t=[t/\tau]\tauδt​=[t/τ]τ, the last observation time up to ttt. The controlled system is

dx(t)=(f(x(t),r(t),t)+u(x(δt),r(t),t))dt+g(x(t),r(t),t) dw(t),x(0)=x0, r(0)=r0,(2.1)dx(t)=\big(f(x(t),r(t),t)+u(x(\delta_t),r(t),t)\big)dt+g(x(t),r(t),t)\,dw(t),\qquad x(0)=x_0,\ r(0)=r_0,\tag{2.1}dx(t)=(f(x(t),r(t),t)+u(x(δt​),r(t),t))dt+g(x(t),r(t),t)dw(t),x(0)=x0​, r(0)=r0​,(2.1)

with f,u:Rn×S×R+→Rnf,u:\mathbb R^n\times S\times\mathbb R_+\to\mathbb R^nf,u:Rn×S×R+​→Rn and g:Rn×S×R+→Rn×mg:\mathbb R^n\times S\times\mathbb R_+\to\mathbb R^{n\times m}g:Rn×S×R+​→Rn×m. The feedback uuu sees the state only at the observation times.

The hypotheses are:

  • Assumption 2.1: f,gf,gf,g locally Lipschitz in xxx, and ∣f(x,i,t)∣≤K1∣x∣|f(x,i,t)|\le K_1|x|∣f(x,i,t)∣≤K1​∣x∣, ∣g(x,i,t)∣≤K2∣x∣|g(x,i,t)|\le K_2|x|∣g(x,i,t)∣≤K2​∣x∣ (∣g∣|g|∣g∣ the trace norm).
  • Assumption 2.2: ∣u(x,i,t)−u(y,i,t)∣≤K3∣x−y∣|u(x,i,t)-u(y,i,t)|\le K_3|x-y|∣u(x,i,t)−u(y,i,t)∣≤K3​∣x−y∣ and u(0,i,t)=0u(0,i,t)=0u(0,i,t)=0.
  • Assumption 3.1: there are U∈C2,1(Rn×S×R+;R+)U\in C^{2,1}(\mathbb R^n\times S\times\mathbb R_+;\mathbb R_+)U∈C2,1(Rn×S×R+​;R+​) and λ1,λ2>0\lambda_1,\lambda_2>0λ1​,λ2​>0 with LU(x,i,t)+λ1∣Ux(x,i,t)∣2≤−λ2∣x∣2\mathcal LU(x,i,t)+\lambda_1|U_x(x,i,t)|^2\le-\lambda_2|x|^2LU(x,i,t)+λ1​∣Ux​(x,i,t)∣2≤−λ2​∣x∣2, where
LU=Ut+Ux[f+u]+12trace⁡[gTUxxg]+∑jγijU(x,j,t).\mathcal LU=U_t+U_x[f+u]+\tfrac12\operatorname{trace}[g^TU_{xx}g]+\sum_j\gamma_{ij}U(x,j,t).LU=Ut​+Ux​[f+u]+21​trace[gTUxx​g]+j∑​γij​U(x,j,t).
  • Condition (3.5): λ2>τK32λ1[2τ(K12+2K32)+K22]\lambda_2>\frac{\tau K_3^2}{\lambda_1}\big[2\tau(K_1^2+2K_3^2)+K_2^2\big]λ2​>λ1​τK32​​[2τ(K12​+2K32​)+K22​] and τ≤14K3\tau\le\frac1{4K_3}τ≤4K3​1​.

In Lean these are Assumption21, Assumption22, C21, LU, Assumption31, Condition35, in the namespace You2015.Asymp; the basis is HybridSetup, the Itô integral IsItoIntegral, the sampling time delta, and solutions SolvesSampledHybridSDE, in the namespace You2015.Shared shared with the companion mission.

Formalization targets

Goal: Theorem 3.4 (almost sure asymptotic stability)

Under the hypotheses above, every solution of (2.1) satisfies

lim⁡t→∞x(t)=0a.s.\lim_{t\to\infty}x(t)=0\quad\text{a.s.}t→∞lim​x(t)=0a.s.

for all x0∈Rnx_0\in\mathbb R^nx0​∈Rn and r0∈Sr_0\in Sr0​∈S. No rate is claimed; the statement is the qualitative convergence of almost every path.

Milestones, in the order the proof uses them

  1. (3.15) E∣x(t)−x(δt)∣2≤2E∫δtt[τ∣f+u(x(δs),⋅)∣2+∣g∣2]ds\mathbb E|x(t)-x(\delta_t)|^2\le2\mathbb E\int_{\delta_t}^t[\tau|f+u(x(\delta_s),\cdot)|^2+|g|^2]dsE∣x(t)−x(δt​)∣2≤2E∫δt​t​[τ∣f+u(x(δs​),⋅)∣2+∣g∣2]ds.
  2. Theorem 3.2 (H∞H_\inftyH∞​-stability): ∫0∞E∣x(s)∣2ds<∞\int_0^\infty\mathbb E|x(s)|^2ds<\infty∫0∞​E∣x(s)∣2ds<∞.
  3. (3.21) E∣x(s)−x(δs)∣2≤3(τK12+K22)1−6τ2K32∫δssE∣x(z)∣2dz+6τ2K321−6τ2K32E∣x(s)∣2\mathbb E|x(s)-x(\delta_s)|^2\le\frac{3(\tau K_1^2+K_2^2)}{1-6\tau^2K_3^2}\int_{\delta_s}^s\mathbb E|x(z)|^2dz+\frac{6\tau^2K_3^2}{1-6\tau^2K_3^2}\mathbb E|x(s)|^2E∣x(s)−x(δs​)∣2≤1−6τ2K32​3(τK12​+K22​)​∫δs​s​E∣x(z)∣2dz+1−6τ2K32​6τ2K32​​E∣x(s)∣2.
  4. (3.23) sup⁡t≥0E∣x(t)∣2<∞\sup_{t\ge0}\mathbb E|x(t)|^2<\inftysupt≥0​E∣x(t)∣2<∞.
  5. ∣E∣x(t2)∣2−E∣x(t1)∣2∣≤C(t2−t1)|\mathbb E|x(t_2)|^2-\mathbb E|x(t_1)|^2|\le C(t_2-t_1)∣E∣x(t2​)∣2−E∣x(t1​)∣2∣≤C(t2​−t1​).
  6. Theorem 3.3: lim⁡t→∞E∣x(t)∣2=0\lim_{t\to\infty}\mathbb E|x(t)|^2=0limt→∞​E∣x(t)∣2=0.
  7. (3.24)–(3.25): E∫0∞∣x(t)∣2dt<∞\mathbb E\int_0^\infty|x(t)|^2dt<\inftyE∫0∞​∣x(t)∣2dt<∞ and lim inf⁡t→∞∣x(t)∣=0\liminf_{t\to\infty}|x(t)|=0liminft→∞​∣x(t)∣=0 a.s.
  8. (3.28): P(∃t:∣x(t)∣≥h)≤C/h2\mathbb P(\exists t:|x(t)|\ge h)\le C/h^2P(∃t:∣x(t)∣≥h)≤C/h2 for h>∣x0∣h>|x_0|h>∣x0​∣.

Significance

The result. Theorem 3.4 says that a controller sampling the state at rate 1/τ1/\tau1/τ makes almost every trajectory of the switching system converge to the equilibrium, with an explicit, checkable bound (3.5) on τ\tauτ. Mean-square convergence (Theorem 3.3) does not imply almost sure convergence in general, and a single trajectory is what an operator observes, so the pathwise statement is the one relevant to a deployed system. Condition (3.5) is stated in terms of the constants of Assumptions 2.1, 2.2 and 3.1, so for a concrete system (Section 6 of the paper) it gives a numerical bound on the observation interval.

Formalizing it. The results are proved in the paper; none of them is machine-checked. Mathlib has real Brownian motion but no Itô integral, no stochastic differential equations and no continuous-time Markov chains. The mission therefore also produces a reusable definition layer: a filtration under the usual conditions, a multidimensional {Ft}\{\mathcal F_t\}{Ft​}-Brownian motion, an {Ft}\{\mathcal F_t\}{Ft​}-Markov chain with a given generator, the L2L^2L2 Itô integral of vector-valued integrands, and the solution notion of an SDE with Markovian switching and a sampled-state delay. A related but different layer exists on Prove2Me for Ethier–Kurtz (EthierKurtz_IsStandardBrownian, EthierKurtz_HasBrownianItoIntegral, EthierKurtz_SolvesBrownianSDE); it has no mode switching and no sampled state, so it cannot express (2.1).

Difficulty

Equation (2.1) is a stochastic differential delay equation with the delay t−δtt-\delta_tt−δt​, which is bounded but jumps at every observation time and has derivative 111 in between. The stability theorems for hybrid delay equations in the literature require a differentiable delay with derivative less than one (Mao–Yuan, p. 285), so they do not apply. Applying LU\mathcal LULU directly to U(x(t),r(t),t)U(x(t),r(t),t)U(x(t),r(t),t) leaves the term Ux[u(x(t))−u(x(δt))]U_x[u(x(t))-u(x(\delta_t))]Ux​[u(x(t))−u(x(δt​))], which has no sign and depends on the path over a whole observation interval, so a Lyapunov function of the current state alone does not close the argument.

For the goal, the natural first idea, deducing almost sure convergence from E∣x(t)∣2→0\mathbb E|x(t)|^2\to0E∣x(t)∣2→0 or from ∫0∞∣x(t)∣2dt<∞\int_0^\infty|x(t)|^2dt<\infty∫0∞​∣x(t)∣2dt<∞ a.s., fails: both are compatible with paths that make ever shorter excursions away from 000. The obstacle is to exclude infinitely many excursions of a fixed size, which neither moment statement controls.

Formalization scope

Conventions committed to in Lean:

  • The state space is EuclideanSpace ℝ (Fin n), so ∣x∣|x|∣x∣ is the Euclidean norm; the explicit constants in (3.5) and (3.21) depend on it. The diffusion ggg is given by its mmm columns and ∣g∣2=∑k∣gk∣2|g|^2=\sum_k|g_k|^2∣g∣2=∑k​∣gk​∣2 (trace norm). Modes are Fin N (0-based). Time is ℝ≥0; time integrals are over subsets of R\mathbb RR at s.toNNReal.
  • Every expectation E∣⋅∣2\mathbb E|\cdot|^2E∣⋅∣2 and every time integral of a nonnegative quantity is a lower Lebesgue integral in [0,∞][0,\infty][0,∞], so a non-integrable process cannot produce a junk value 000.
  • "The solution of (2.1)" is read as every process satisfying the solution definition: progressively measurable, almost surely continuous paths, E∣x(t)∣2<∞\mathbb E|x(t)|^2<\inftyE∣x(t)∣2<∞ for each ttt, and for each ttt, almost surely, the integral equation with Itô integrals in the L2L^2L2 sense. Existence and uniqueness (cited from Mao–Yuan on p. 908) are not asserted.
  • "An mmm-dimensional Brownian motion" and "a Markov chain with generator Γ\GammaΓ" are read in the Mao–Yuan framework the paper cites: an {Ft}\{\mathcal F_t\}{Ft​}-Brownian motion with independent coordinates and increments independent of the past, and an {Ft}\{\mathcal F_t\}{Ft​}-Markov chain with transition matrix etΓe^{t\Gamma}etΓ. The usual conditions are kept as hypotheses.
  • "Locally Lipschitz" is uniform in the mode and time on each ball. C2,1C^{2,1}C2,1 carries its derivatives Ut,Ux,UxxU_t,U_x,U_{xx}Ut​,Ux​,Uxx​ as witnesses tied to UUU by derivative relations and joint continuity.
  • "τ>0\tau>0τ>0 sufficiently small for (3.5)" means every τ>0\tau>0τ>0 satisfying both inequalities of (3.5). U,λ1,λ2,τU,\lambda_1,\lambda_2,\tauU,λ1​,λ2​,τ are data of each statement. The paper's "CCC denotes a positive constant" is an existential chosen after x0x_0x0​, r0r_0r0​ and the solution, and before the time variables and hhh.
  • (3.15) and (3.21) are stated under fewer hypotheses than the surrounding proof has (Assumptions 2.1, 2.2, τ>0\tau>0τ>0, and for (3.21) τ≤1/(4K3)\tau\le1/(4K_3)τ≤1/(4K3​)), because their derivations use no more. Misprints on the page (for example g(x,i,s)=f(x,i,0)g(x,i,s)=f(x,i,0)g(x,i,s)=f(x,i,0) on p. 909 and the swapped definitions of ∨,∧\vee,\wedge∨,∧ on p. 907) are not formalized.

A trivializing formalization is ruled out: the expectations are not Bochner integrals (which vanish for non-integrable integrands), the solution notion admits the true solution and requires path continuity, the derivative witnesses of UUU are tied to UUU, and a sorry-free check shows that the data hypotheses (Assumptions 2.1, 2.2, 3.1, C2,1C^{2,1}C2,1, (3.5)) are satisfiable, for example by n=m=N=1n=m=N=1n=m=N=1, f=g=0f=g=0f=g=0, u(x)=−xu(x)=-xu(x)=−x, U=∣x∣2U=|x|^2U=∣x∣2, λ1=1/4\lambda_1=1/4λ1​=1/4, λ2=1\lambda_2=1λ2​=1, τ=1/10\tau=1/10τ=1/10.

Welcome contributions: the Itô isometry and Itô's formula for the L2L^2L2 integral defined here, a generalized Itô formula for functions of a Markov-modulated Itô process, and Doob's maximal inequality in continuous time. These are reusable far beyond this mission. Section 4 of the paper (exponential stability) is a separate mission of the same series.

Selected references

  • S. You, W. Liu, J. Lu, X. Mao, Q. Qiu, Stabilization of Hybrid Systems by Feedback Control Based on Discrete-Time State Observations, SIAM J. Control Optim. 53(2), 905–925, 2015. https://doi.org/10.1137/140985779
  • X. Mao, C. Yuan, Stochastic Differential Equations with Markovian Switching, Imperial College Press, 2006. https://doi.org/10.1142/p473
  • X. Mao, Stabilization of continuous-time hybrid stochastic differential equations by discrete-time feedback control, Automatica 49(12), 3677–3681, 2013. https://doi.org/10.1016/j.automatica.2013.09.005
13 thms1 active userReviewed
Control TheoryOperations ResearchOptimization+1·Captain: mikedeng1

A General Stochastic Maximum Principle for Optimal Control Problems: The Maximum Principle with First- and Second-Order Adjoint ProcessesResearch Paper

Motivation

Pontryagin's maximum principle gives necessary conditions for optimality in deterministic optimal control: along an optimal trajectory, the optimal control maximizes (or minimizes) a Hamiltonian built from an adjoint process. For a system driven by Brownian noise the analogous statement was open in full generality for two decades. The difficulty appears exactly when the diffusion coefficient depends on the control and the control domain is not convex, the situation of controlled volatility in finance, of controlled noise intensity in engineering, and of any problem whose admissible actions form a discrete or otherwise nonconvex set.

Shige Peng's 1990 paper (SIAM J. Control Optim. 28(4)) closed that case. It introduced the second-order adjoint process and a second-order variational inequality, and it is the starting point of the modern theory of stochastic Hamiltonian systems and of backward stochastic differential equations as a tool in control.

Timeline.

  • 1972: Kushner obtains necessary conditions for diffusions whose diffusion coefficient does not depend on the control (SIAM J. Control 10).
  • 1973–1978: Bismut introduces the adjoint equation as a linear backward stochastic differential equation and develops duality methods (SIAM Review 20).
  • Early 1980s: Bensoussan and Haussmann prove maximum principles for convex control domains or control-independent diffusion, using the first-order adjoint equation only.
  • 1990: Peng proves the general principle, with control-dependent diffusion and an arbitrary nonempty control domain (this mission). In the same year Pardoux and Peng prove existence and uniqueness for nonlinear backward SDEs (Systems Control Lett. 14).
  • 1999: Yong and Zhou give a textbook account of the theory (Springer).

Setting

Let (Ω,F,P)(\Omega,\mathcal F,P)(Ω,F,P) be a probability space carrying a standard ddd-dimensional Wiener process B=(B1,…,Bd)B=(B^1,\dots,B^d)B=(B1,…,Bd), and let Ft=σ{B(s);0≤s≤t}\mathcal F^t=\sigma\{B(s);0\le s\le t\}Ft=σ{B(s);0≤s≤t} be its natural filtration. Fix a horizon T>0T>0T>0, an initial state x0∈Rnx_0\in\mathbb R^nx0​∈Rn and a nonempty control domain U⊆RkU\subseteq\mathbb R^kU⊆Rk. The data are

g:Rn×Rk→Rn,σ=(σ1,…,σd), σj:Rn×Rk→Rn,l:Rn×Rk→R,h:Rn→R.g:\mathbb R^n\times\mathbb R^k\to\mathbb R^n,\quad \sigma=(\sigma^1,\dots,\sigma^d),\ \sigma^j:\mathbb R^n\times\mathbb R^k\to\mathbb R^n,\quad l:\mathbb R^n\times\mathbb R^k\to\mathbb R,\quad h:\mathbb R^n\to\mathbb R .g:Rn×Rk→Rn,σ=(σ1,…,σd), σj:Rn×Rk→Rn,l:Rn×Rk→R,h:Rn→R.

An admissible control vvv is a progressively measurable UUU-valued process with sup⁡t≤TE∣v(t)∣m<∞\sup_{t\le T}E|v(t)|^m<\inftysupt≤T​E∣v(t)∣m<∞ for every m≥1m\ge1m≥1. Its trajectory solves the state equation

dx(t)=g(x(t),v(t)) dt+∑j=1dσj(x(t),v(t)) dBj(t),x(0)=x0,dx(t)=g(x(t),v(t))\,dt+\sum_{j=1}^d\sigma^j(x(t),v(t))\,dB^j(t),\qquad x(0)=x_0,dx(t)=g(x(t),v(t))dt+j=1∑d​σj(x(t),v(t))dBj(t),x(0)=x0​,

and its cost is J(v)=E∫0Tl(x(t),v(t)) dt+E h(x(T))J(v)=E\int_0^Tl(x(t),v(t))\,dt+E\,h(x(T))J(v)=E∫0T​l(x(t),v(t))dt+Eh(x(T)). A pair (y,u)(y,u)(y,u) is optimal when J(u)≤J(v)J(u)\le J(v)J(u)≤J(v) for every admissible vvv.

Assumption (3): g,σ,l,hg,\sigma,l,hg,σ,l,h are C2C^2C2 in xxx, jointly continuous in (x,v)(x,v)(x,v) together with their first and second xxx-derivatives; gx,gxx,σx,σxx,lxx,hxxg_x,g_{xx},\sigma_x,\sigma_{xx},l_{xx},h_{xx}gx​,gxx​,σx​,σxx​,lxx​,hxx​ are bounded; and g,σ,lx,hxg,\sigma,l_x,h_xg,σ,lx​,hx​ grow at most like C(1+∣x∣+∣v∣)C(1+|x|+|v|)C(1+∣x∣+∣v∣).

The Hamiltonian is H(x,v,p,K)=l(x,v)+(p,g(x,v))+∑j(Kj,σj(x,v))H(x,v,p,K)=l(x,v)+(p,g(x,v))+\sum_j(K_j,\sigma^j(x,v))H(x,v,p,K)=l(x,v)+(p,g(x,v))+∑j​(Kj​,σj(x,v)). The first-order adjoint process (p,K)(p,K)(p,K) solves the backward equation

−dp=[gx∗p+∑jσxj∗Kj+lx]dt−∑jKj dBj,p(T)=hx(y(T)),-dp=\Big[g_x^*p+\sum_j\sigma_x^{j*}K_j+l_x\Big]dt-\sum_jK_j\,dB^j,\qquad p(T)=h_x(y(T)),−dp=[gx∗​p+j∑​σxj∗​Kj​+lx​]dt−j∑​Kj​dBj,p(T)=hx​(y(T)),

and the second-order adjoint process (P,Q)(P,Q)(P,Q), symmetric-matrix valued, solves

−dP=[gx∗P+Pgx+∑jσxj∗Pσxj+∑jσxj∗Qj+∑jQjσxj+Hxx]dt−∑jQj dBj,P(T)=hxx(y(T)),-dP=\Big[g_x^*P+Pg_x+\sum_j\sigma_x^{j*}P\sigma_x^j+\sum_j\sigma_x^{j*}Q_j+\sum_jQ_j\sigma_x^j+H_{xx}\Big]dt-\sum_jQ_j\,dB^j,\qquad P(T)=h_{xx}(y(T)),−dP=[gx∗​P+Pgx​+j∑​σxj∗​Pσxj​+j∑​σxj∗​Qj​+j∑​Qj​σxj​+Hxx​]dt−j∑​Qj​dBj,P(T)=hxx​(y(T)),

with all coefficients evaluated along (y(t),u(t))(y(t),u(t))(y(t),u(t)) and both solutions adapted to Ft\mathcal F^tFt.

Formalization targets

Goal: Theorem 3 (p. 975)

If (y,u)(y,u)(y,u) is optimal, then adjoint processes (p,K)(p,K)(p,K) and (P,Q)(P,Q)(P,Q) exist in LF2L^2_{\mathcal F}LF2​, solving the two equations above, such that for every v∈Uv\in Uv∈U, for almost every τ∈[0,T]\tau\in[0,T]τ∈[0,T], almost surely,

H(y,v,p,K−Pσ(y,u))+12tr⁡(σσ∗(y,v)P) ≥ H(y,u,p,K−Pσ(y,u))+12tr⁡(σσ∗(y,u)P),H\big(y,v,p,K-P\sigma(y,u)\big)+\tfrac12\operatorname{tr}\big(\sigma\sigma^*(y,v)P\big)\ \ge\ H\big(y,u,p,K-P\sigma(y,u)\big)+\tfrac12\operatorname{tr}\big(\sigma\sigma^*(y,u)P\big),H(y,v,p,K−Pσ(y,u))+21​tr(σσ∗(y,v)P) ≥ H(y,u,p,K−Pσ(y,u))+21​tr(σσ∗(y,u)P),

all evaluated at time τ\tauτ.

Milestones

  1. Lemma 1: the spike-perturbed state equals y+y1+y2y+y_1+y_2y+y1​+y2​ up to o(ε2)o(\varepsilon^2)o(ε2) in mean square, where y1,y2y_1,y_2y1​,y2​ solve the first- and second-order variational equations (5), (6).
  2. Lemma 2: the cost expansion (11) is ≥o(ε)\ge o(\varepsilon)≥o(ε) at an optimal control.
  3. Eq. (13): existence and uniqueness of (p,K)(p,K)(p,K) as a Riesz representer.
  4. Eq. (14): the cost expansion in Hamiltonian form is ≥o(ε)\ge o(\varepsilon)≥o(ε).
  5. Eq. (17): existence and uniqueness of (P,Q)(P,Q)(P,Q) as a Riesz representer.
  6. Eq. (18): the variational inequality for these representers.
  7. Eq. (19): (p,K)(p,K)(p,K) is the unique solution of the first-order adjoint equation.
  8. Eq. (20): (P,Q)(P,Q)(P,Q) solves the second-order adjoint equation.

Significance

The result. Theorem 3 is the necessary condition for optimal control of diffusions in its general form. When σ\sigmaσ does not depend on the control the trace terms cancel and it reduces to the classical first-order principle. When the control enters the diffusion, the first-order condition is false in general, and the second-order adjoint PPP is the correction. The theorem underlies stochastic linear-quadratic theory, the verification of optimal portfolio and volatility-control policies, and the relation between the maximum principle and the Hamilton–Jacobi–Bellman equation.

Formalizing it. The theorem is classical and proved; none of it is machine-checked. Mathlib has real Brownian motion but no stochastic integral, no SDE and no backward SDE. This mission produces the first formal statements on the platform of a controlled SDE, of a backward SDE and of the maximum principle, together with a precise definition layer (Itô integral, Itô process, BSDE solution) that later missions can reuse, e.g. for Peng's endpoint-constrained principle (§6 of the paper) or for the existence theory of BSDEs. A formal proof would also pin down the approximation arguments the paper leaves to the reader.

Difficulty

The obvious route perturbs the optimal control convexly, u+ε(v−u)u+\varepsilon(v-u)u+ε(v−u), and differentiates the cost. That needs UUU convex. For nonconvex UUU one uses a spike variation on a time interval of length ε\varepsilonε. For deterministic systems the state then moves by O(ε)O(\varepsilon)O(ε) and a first-order expansion suffices. With control-dependent diffusion the stochastic integral over the spike interval moves the state by order ε\sqrt\varepsilonε​ in L2L^2L2, so the first-order variational equation leaves an error of the same order as the effect being measured. Second-order terms in the state enter the cost at order ε\varepsilonε, and they are quadratic, so they cannot be handled by a single linear adjoint. The second-order expansion, the matrix-valued adjoint that represents the quadratic term, and the identification of both adjoints with backward SDEs are where the work lies. On the formal side, none of the stochastic calculus exists in Mathlib: the Itô isometry, Itô's formula for matrix-valued processes, moment estimates for linear SDEs and the martingale representation behind the backward equations all have to be built.

Formalization scope

Conventions committed to in Lean:

  • States, controls and noise are Fin n → ℝ, Fin k → ℝ, Fin d → ℝ with the sup norm; matrices are Matrix (Fin n) (Fin n) ℝ. Time is ℝ≥0, and time integrals are over [0, t] ⊂ ℝ.
  • The Wiener process is Rd\mathbb R^dRd-valued: the paper's "RnR^nRn-valued standard Wiener process" (p. 967) is a misprint, since σ(x,v)∈L(Rd,Rn)\sigma(x,v)\in\mathcal L(R^d,R^n)σ(x,v)∈L(Rd,Rn). Coordinates are independent real Brownian motions (Mathlib's IsBrownianReal).
  • The filtration is the natural filtration of BBB, not completed, as on p. 967.
  • "Adapted", for processes integrated in dtdtdt, is read as progressively measurable.
  • The Itô integral is a relation (an L2L^2L2 limit of elementary integrals, as in Ikeda–Watanabe), not an operator. SDE and BSDE solutions hold "for every ttt, almost surely", with sup⁡tE∣x(t)∣2<∞\sup_tE|x(t)|^2<\inftysupt​E∣x(t)∣2<∞ for forward solutions and LF2L^2_{\mathcal F}LF2​ membership for backward ones.
  • Optimality is among admissible controls of finite cost, and the optimal cost is finite; the paper never states finiteness, and a cost can be +∞+\infty+∞ under (3).
  • Lemma 1 is stated with o(ε2)o(\varepsilon^2)o(ε2) where the page prints "≤Cε2\le C\varepsilon^2≤Cε2" in (4). The proof (via (10)) establishes o(ε2)o(\varepsilon^2)o(ε2), and Lemma 2 needs it. Lemma 1, (13), (17), (19) and (20) are stated for any admissible pair, since their proofs do not use optimality.
  • "≥o(ε)\ge o(\varepsilon)≥o(ε)" means: some r(ε)=o(ε)r(\varepsilon)=o(\varepsilon)r(ε)=o(ε) as ε→0+\varepsilon\to0^+ε→0+ bounds the left side from below for small ε\varepsilonε. "∀v∈U\forall v\in U∀v∈U, a.e., a.s." quantifies vvv first, then τ\tauτ, then ω\omegaω.
  • PPP and QjQ_jQj​ are symmetric-valued (Rn,nR^{n,n}Rn,n is the space of symmetric matrices, p. 973). Stochastic integrals against the matrix σ\sigmaσ or QQQ are sums over the columns, ∑j(⋅)j dBj\sum_j(\cdot)_j\,dB^j∑j​(⋅)j​dBj.

Trivializing formalizations ruled out. Without adaptedness the backward equations have pathwise solutions with K=0K=0K=0 and the goal would be free; every adjoint and every solution is required to be progressive for the natural filtration of BBB. The hypotheses are satisfiable: a sorry-free check shows that the zero problem has an optimal pair and that the Itô relation holds for the zero integrand.

Infrastructure needed, and reusable. The Itô integral and isometry, Itô's formula (vector and matrix forms), existence, uniqueness and moment estimates for linear SDEs with bounded coefficients, Riesz representation in LF2L^2_{\mathcal F}LF2​, and existence and uniqueness for linear BSDEs (via martingale representation for the Brownian filtration). All of these are reusable well beyond this mission; contributions of any of them, as separate theorems, are welcome. Related platform definitions: the Ethier–Kurtz series (EthierKurtz_HasBrownianItoIntegral, EthierKurtz_SolvesBrownianSDE) formalizes an Itô integral by dyadic step approximation and uncontrolled SDEs over a completed filtration. The deterministic Pontryagin principle appears in Vector Space Methods XII and Dynamic Programming and Optimal Control III.

Selected references

  • S. Peng, A General Stochastic Maximum Principle for Optimal Control Problems, SIAM J. Control Optim. 28(4), 966–979, 1990. https://doi.org/10.1137/0328054
  • H. J. Kushner, Necessary Conditions for Continuous Parameter Stochastic Optimization Problems, SIAM J. Control 10(3), 550–565, 1972. https://doi.org/10.1137/0310041
  • J.-M. Bismut, An Introductory Approach to Duality in Optimal Stochastic Control, SIAM Review 20(1), 62–78, 1978. https://doi.org/10.1137/1020004
  • E. Pardoux and S. Peng, Adapted Solution of a Backward Stochastic Differential Equation, Systems & Control Letters 14(1), 55–61, 1990. https://doi.org/10.1016/0167-6911(90)90082-6
  • J. Yong and X. Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations, Springer, 1999. https://doi.org/10.1007/978-1-4612-1466-3
12 thms1 active userReviewed
Bandit AlgorithmsMachine LearningOperations Research·Captain: mikedeng1

Online Network Revenue Management Using Thompson Sampling: Bayesian Regret of TS-fixedResearch Paper

Motivation

A retailer who sells several products from shared, non-replenishable inventory over a finite season must set prices without knowing how demand responds to them. Every price posted is both a sale and an experiment. This is the network revenue management problem with demand learning, and it sits between two literatures: dynamic pricing with inventory, where demand is known and the fluid linear program of Gallego and van Ryzin (1997) is the standard benchmark, and multi-armed bandits, where learning is the whole problem but there are no resource constraints.

Ferreira, Simchi-Levi and Wang (Oper. Res. 2018) combine Thompson sampling with a linear-programming step: sample a demand model from the posterior, solve the fluid LP for that model, and randomize prices according to its solution. The same paper extends the scheme to continuous price sets, contextual pricing and bandits with knapsacks.

Timeline of the relevant results:

  • 1997: Gallego and van Ryzin introduce the fluid LP upper bound for network revenue management with known demand.
  • 2012: Besbes and Zeevi give a non-Bayesian network pricing algorithm with worst-case regret O(K5/3T2/3log⁡T)O(K^{5/3}T^{2/3}\sqrt{\log T})O(K5/3T2/3logT​).
  • 2013: Badanidiyuru, Kleinberg and Slivkins (bandits with knapsacks) give worst-case regret O(KTlog⁡T)O(\sqrt{KT\log T})O(KTlogT​).
  • 2013–2014: Bubeck and Liu and Russo and Van Roy give prior-free Bayesian regret bounds for Thompson sampling in unconstrained bandits.
  • 2018: Ferreira, Simchi-Levi and Wang prove the O(TKlog⁡K)O(\sqrt{TK\log K})O(TKlogK​) Bayesian regret bound for TS-fixed (Theorem 1), the target of this mission.

Setting

There are NNN products and MMM resources. One unit of product iii consumes aij≥0a_{ij}\ge0aij​≥0 units of resource jjj, and resource jjj starts with inventory Ij≥0I_j\ge0Ij​≥0 that is never replenished. The season has TTT periods. In each period the retailer posts one of KKK price vectors pk=(p1k,…,pNk)p_k=(p_{1k},\dots,p_{Nk})pk​=(p1k​,…,pNk​) or a shut-off price p∞p_\inftyp∞​ under which demand is zero.

Given the posted price pkp_kpk​, the demand vector D(t)∈R+ND(t)\in\mathbb R^N_+D(t)∈R+N​ has law F(⋅ ;pk,θ)F(\cdot\,;p_k,\theta)F(⋅;pk​,θ), where θ∈Θ\theta\in\Thetaθ∈Θ is unknown and drawn from a known, arbitrary prior μ0\mu_0μ0​. Demand is independent of the past given the posted price and θ\thetaθ, and is bounded: Di(t)∈[0,dˉi]D_i(t)\in[0,\bar d_i]Di​(t)∈[0,dˉi​]. Write dik(ρ)d_{ik}(\rho)dik​(ρ) for the mean demand of product iii under pkp_kpk​ and parameter ρ\rhoρ, and d=d(θ)d=d(\theta)d=d(θ).

When inventory covers all demand, all demand is sold. Otherwise the satisfied demand D~(t)\tilde D(t)D~(t) satisfies 0≤D~i(t)≤Di(t)0\le\tilde D_i(t)\le D_i(t)0≤D~i​(t)≤Di​(t), leaves every inventory nonnegative, and leaves at least one resource at zero; no other rule is imposed. Revenue is Rev(T)=∑t∑iD~i(t)Pi(t)\mathrm{Rev}(T)=\sum_t\sum_i\tilde D_i(t)P_i(t)Rev(T)=∑t​∑i​D~i​(t)Pi​(t).

For a mean-demand matrix ddd and capacities cj=Ij/Tc_j=I_j/Tcj​=Ij​/T, the linear program LP(d)\mathrm{LP}(d)LP(d) is

max⁡x≥0 ∑k=1K(∑i=1Npikdik)xks.t.∑k=1K(∑i=1Naijdik)xk≤cj  ∀j,∑k=1Kxk≤1,\max_{x\ge0}\ \sum_{k=1}^K\Bigl(\sum_{i=1}^N p_{ik}d_{ik}\Bigr)x_k\quad\text{s.t.}\quad\sum_{k=1}^K\Bigl(\sum_{i=1}^N a_{ij}d_{ik}\Bigr)x_k\le c_j\ \ \forall j,\qquad\sum_{k=1}^K x_k\le1,x≥0max​ k=1∑K​(i=1∑N​pik​dik​)xk​s.t.k=1∑K​(i=1∑N​aij​dik​)xk​≤cj​  ∀j,k=1∑K​xk​≤1,

with optimal value OPT(d)\mathrm{OPT}(d)OPT(d).

TS-fixed (Algorithm 1): in each period, sample θ(t)\theta(t)θ(t) from the posterior of θ\thetaθ given the history of posted prices and observed demands; let x(t)x(t)x(t) be an optimal solution of LP(d(θ(t)))\mathrm{LP}(d(\theta(t)))LP(d(θ(t))); post pkp_kpk​ with probability xk(t)x_k(t)xk​(t) and p∞p_\inftyp∞​ with the remaining probability; observe demand and update the posterior.

Finally pmax⁡=max⁡k∑ipikdˉip_{\max}=\max_k\sum_ip_{ik}\bar d_ipmax​=maxk​∑i​pik​dˉi​ and pmax⁡j=max⁡i:aij≠0, kpik/aijp^j_{\max}=\max_{i:a_{ij}\neq0,\,k}p_{ik}/a_{ij}pmaxj​=maxi:aij​=0,k​pik​/aij​.

Formalization targets

Goal: Theorem 1 against the LP benchmark

For K≥2K\ge2K≥2, T≥1T\ge1T≥1, every prior, every bounded demand family, every admissible fulfilment rule and every run of TS-fixed,

E[OPT(d)]⋅T−E[Rev(T)] ≤ (18 pmax⁡+37∑i=1N∑j=1Mpmax⁡jaijdˉi)TKlog⁡K.\mathbb E\bigl[\mathrm{OPT}(d)\bigr]\cdot T-\mathbb E\bigl[\mathrm{Rev}(T)\bigr]\ \le\ \Bigl(18\,p_{\max}+37\sum_{i=1}^N\sum_{j=1}^M p^j_{\max}a_{ij}\bar d_i\Bigr)\sqrt{TK\log K}.E[OPT(d)]⋅T−E[Rev(T)] ≤ (18pmax​+37i=1∑N​j=1∑M​pmaxj​aij​dˉi​)TKlogK​.

The paper prints this bound for BayesRegret(T)=E[Rev∗(T)]−E[Rev(T)]\mathrm{BayesRegret}(T)=\mathbb E[\mathrm{Rev}^*(T)]-\mathbb E[\mathrm{Rev}(T)]BayesRegret(T)=E[Rev∗(T)]−E[Rev(T)], where Rev∗\mathrm{Rev}^*Rev∗ is the revenue of the optimal policy that knows θ\thetaθ; see Formalization scope for why the LP benchmark is stated instead.

Milestones

The article states Theorem 1 and says that its proof is in the online appendix (Supplemental Material at the DOI). The article itself contains no numbered lemma. The milestone list is therefore empty; the appendix's lemmas will be added as milestones once the appendix is held.

Significance

The bound is prior-free and has explicit constants that depend only on prices, consumption rates and demand bounds. Its dependence on TTT matches the Ω(KT)\Omega(\sqrt{KT})Ω(KT​) lower bound for Bayesian regret in unconstrained bandits with rewards in [0,1][0,1][0,1], a special case of the model with no inventory constraints (Bubeck and Cesa-Bianchi 2012, Theorem 3.5). It shows that the posterior-sampling principle survives the addition of resource constraints, lost sales and randomized LP-based pricing, and it is the template for the paper's later results (TS-update, contextual pricing, bandits with knapsacks).

The theorem is proved on paper but, as far as a platform search shows, not formalized anywhere. The platform has a formal proof of the unconstrained Bayesian Thompson sampling bound knlog⁡k/2\sqrt{kn\log k/2}knlogk/2​ (BanditAlgorithm.thompson_sampling_bayesian_regret, Lattimore–Szepesvári Theorem 36.5) and an open single-product deterministic upper bound in revenue management (RevenueManagement.deterministic_upper_bound). Neither has inventory, an LP subroutine, or lost sales. A formal proof here would supply the first machine-checked analysis of Thompson sampling under resource constraints and would check the paper's constants.

Difficulty

In an unconstrained bandit, Thompson sampling's regret reduces to a sum of per-period gaps between an upper confidence bound and the sampled reward, because the sampled optimal arm and the true optimal arm are identically distributed given the history. Here the action is a randomized mixture x(t)x(t)x(t) from an LP, the reward is not additive in the prices chosen, and revenue is lost when inventory runs out. Two quantities must be controlled: the revenue the algorithm would collect if all demand could be served, and the revenue lost to stock-outs. The second depends on the random time at which each resource is exhausted under a pricing rule that was optimized for a sampled, not the true, demand, and on an arbitrary fulfilment rule once some resource is empty. Standard bandit arguments do not bound such lost sales, which are a nonlinear function of the whole trajectory.

Formalization scope

Lean representation. Products, resources and price vectors are indexed by Fin N, Fin M, Fin K; the posted price is an Option (Fin K) with none the shut-off price. Periods are 0-based (t=0,…,T−1t=0,\dots,T-1t=0,…,T−1 stands for the paper's 1,…,T1,\dots,T1,…,T). Θ\ThetaΘ is a standard Borel space with a probability measure μ0\mu_0μ0​; demand is a Markov kernel FFF from Θ×\Theta\timesΘ×Fin K to RN\mathbb R^NRN, bounded in [0,dˉi][0,\bar d_i][0,dˉi​] for every parameter. A run of TS-fixed is a family of random variables on a probability space satisfying, almost surely and via conditional expectations: θ∼μ0\theta\sim\mu_0θ∼μ0​; the posterior-sampling property of θ(t)\theta(t)θ(t) given everything before period ttt; the price draw with probabilities x(θ(t))x(\theta(t))x(θ(t)) for a measurable optimal LP selection xxx; the demand law given the past, θ(t)\theta(t)θ(t) and the posted price; and fulfilment rules (a)/(b). The logarithm is natural. Prices, consumption and inventory are nonnegative (implicit in the paper). OPT(d)\mathrm{OPT}(d)OPT(d) is a supremum over a nonempty bounded feasible set, so it has no junk value.

Corrections to the printed statement.

  1. K≥2K\ge2K≥2 is added. At K=1K=1K=1 the printed right-hand side is 000, yet on a one-price instance with Bernoulli(0.8)(0.8)(0.8) demand, I=T/2I=T/2I=T/2 and a point-mass prior, TS-fixed loses about 0.2pT0.2p\sqrt T0.2pT​ in expectation.
  2. The LP benchmark replaces E[Rev∗(T)]\mathbb E[\mathrm{Rev}^*(T)]E[Rev∗(T)]. Section 3.1.1 bounds E[Rev∗(T)∣d]\mathbb E[\mathrm{Rev}^*(T)\mid d]E[Rev∗(T)∣d] by OPT(d)⋅T\mathrm{OPT}(d)\cdot TOPT(d)⋅T, citing Gallego–van Ryzin. Under the paper's fulfilment rule this fails when products use disjoint resources: with two products, I=(T,1)I=(T,1)I=(T,1), p1=(1,0)p_1=(1,0)p1​=(1,0), p2=(1/2,0)p_2=(1/2,0)p2​=(1/2,0) and deterministic demand (1,1)(1,1)(1,1), the known-θ\thetaθ policy earns at least TTT while OPT(d)⋅T=1\mathrm{OPT}(d)\cdot T=1OPT(d)⋅T=1. The paper states that its proof bounds the gap to "the LP benchmark defined in Section 3.1.1" (p. 1594), and the last display of Section 3.1.1 bounds BayesRegret(T)\mathrm{BayesRegret}(T)BayesRegret(T) by exactly E[OPT(d)]⋅T−E[Rev(T)]\mathbb E[\mathrm{OPT}(d)]\cdot T-\mathbb E[\mathrm{Rev}(T)]E[OPT(d)]⋅T−E[Rev(T)]. Wherever the Gallego–van Ryzin bound holds, the corrected goal implies the printed one.

Ruled out. A bound for the "ideal" revenue ∑iDi(t)Pi(t)\sum_iD_i(t)P_i(t)∑i​Di​(t)Pi​(t) instead of the satisfied revenue, or for an arbitrary policy whose prices are merely close to the LP solution, is not Theorem 1; the goal carries the full TS-fixed run and the lost-sales accounting.

Infrastructure needed. Posterior-sampling identities for general (standard Borel) priors, a Hoeffding/Azuma-type concentration for bounded demand along the price-selection process, LP sensitivity with respect to the mean-demand matrix, and a pathwise bound on lost sales under an arbitrary fulfilment rule. The LP and fluid-benchmark definitions are reusable for later missions on TS-update (Theorem 2), contextual pricing (Theorem 4) and bandits with knapsacks (Theorem 5). Contributions welcome: proofs of the goal, and formal statements of the online appendix's lemmas.

Selected references

  • K. J. Ferreira, D. Simchi-Levi, H. Wang, Online Network Revenue Management Using Thompson Sampling, Operations Research 66(6):1586–1602, 2018. https://doi.org/10.1287/opre.2018.1755
  • G. Gallego, G. van Ryzin, A Multiproduct Dynamic Pricing Problem and Its Applications to Network Yield Management, Operations Research 45(1):24–41, 1997. https://doi.org/10.1287/opre.45.1.24
  • O. Besbes, A. Zeevi, Blind Network Revenue Management, Operations Research 60(6):1537–1550, 2012. https://doi.org/10.1287/opre.1120.1057
  • A. Badanidiyuru, R. Kleinberg, A. Slivkins, Bandits with Knapsacks, FOCS 2013. https://arxiv.org/abs/1305.2545
  • S. Bubeck, C.-Y. Liu, Prior-free and Prior-dependent Regret Bounds for Thompson Sampling, NeurIPS 2013. https://arxiv.org/abs/1311.0466
  • D. Russo, B. Van Roy, Learning to Optimize via Posterior Sampling, Mathematics of Operations Research 39(4):1221–1243, 2014. https://doi.org/10.1287/moor.2014.0650
  • S. Bubeck, N. Cesa-Bianchi, Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems, Foundations and Trends in Machine Learning 5(1), 2012. https://arxiv.org/abs/1204.5721
  • T. Lattimore, C. Szepesvári, Bandit Algorithms, Cambridge University Press, 2020, Chapter 36. https://doi.org/10.1017/9781108571401
3 thms1 active userReviewed
Operations ResearchOptimizationTheoretical Computer Science·Captain: mikedeng1

An Optimal On-Line Algorithm for Metrical Task System 2: The Randomized Competitive Ratio of the Uniform Task System Lies Between H(n) and 2H(n)Research Paper

Motivation

Metrical task systems, introduced by Borodin, Linial and Saks (J. ACM 39(4), 1992), are a common abstraction of on-line problems in which a server occupies one of finitely many states, pays a cost for each task depending on its current state, and may pay a transition cost to change state first. Paging, list update and the kkk-server problem all fit into this framework. The paper's first main result is that every deterministic on-line algorithm on an nnn-state metrical task system has competitive ratio at least 2n−12n-12n−1, and that 2n−12n-12n−1 is attained. The lower bound comes from an adversary that always charges the state the algorithm currently occupies. That adversary needs to know the algorithm's state, which suggests that randomization can help.

Section 7 of the paper makes this precise for the simplest system, the uniform task system, in which all transitions cost 111. There the randomized competitive ratio against an oblivious adversary is between H(n)H(n)H(n) and 2H(n)2H(n)2H(n), where H(n)=1+12+⋯+1nH(n)=1+\tfrac12+\cdots+\tfrac1nH(n)=1+21​+⋯+n1​ is between ln⁡n\ln nlnn and 1+ln⁡n1+\ln n1+lnn. This was the first logarithmic bound for a task system.

Timeline:

  • 1985: Sleator and Tarjan introduce competitive analysis for list update and paging (CACM 28(2)).
  • 1987/1992: Borodin, Linial and Saks define metrical task systems, prove the deterministic ratio 2n−12n-12n−1, and prove H(n)≤wˉ≤2H(n)H(n)\le\bar w\le 2H(n)H(n)≤wˉ≤2H(n) for the uniform system (conference version STOC 1987; journal version cited above).
  • 1991: Fiat, Karp, Luby, McGeoch, Sleator and Young prove the analogous 2Hk2H_k2Hk​ upper bound for randomized paging (J. Algorithms 12(4)).

Setting

A task system (S,d)(S,d)(S,d) is a finite set SSS of nnn states with a transition-cost matrix ddd: d(i,i)=0d(i,i)=0d(i,i)=0, d(i,j)>0d(i,j)>0d(i,j)>0 for i≠ji\ne ji=j, and d(i,k)≤d(i,j)+d(j,k)d(i,k)\le d(i,j)+d(j,k)d(i,k)≤d(i,j)+d(j,k). In the uniform task system, d(i,j)=1d(i,j)=1d(i,j)=1 for all i≠ji\neq ji=j. A task is a vector T∈R≥0ST\in\mathbb R_{\ge0}^ST∈R≥0S​ of processing costs. Given an initial state s0s_0s0​ and tasks T=T1⋯Tm\mathbf T=T^1\cdots T^mT=T1⋯Tm, a schedule is σ:{0,…,m}→S\sigma:\{0,\dots,m\}\to Sσ:{0,…,m}→S with σ(0)=s0\sigma(0)=s_0σ(0)=s0​, of cost

c(T;σ)=∑i=1md(σ(i−1),σ(i))+∑i=1mTi(σ(i)).c(\mathbf T;\sigma)=\sum_{i=1}^m d(\sigma(i-1),\sigma(i))+\sum_{i=1}^m T^i(\sigma(i)).c(T;σ)=i=1∑m​d(σ(i−1),σ(i))+i=1∑m​Ti(σ(i)).

The off-line optimum c0(T)c_0(\mathbf T)c0​(T) is the least cost over all schedules.

A deterministic on-line algorithm chooses σ(i)\sigma(i)σ(i) from s0s_0s0​ and T1,…,TiT^1,\dots,T^iT1,…,Ti. A randomized on-line algorithm RRR chooses σ(i)\sigma(i)σ(i) at random, with a distribution that depends on s0s_0s0​, on T1,…,TiT^1,\dots,T^iT1,…,Ti and on the states σ(0),…,σ(i−1)\sigma(0),\dots,\sigma(i-1)σ(0),…,σ(i−1) already visited. The task sequence is fixed before any random choice is made (an oblivious adversary). With pr(σ∣T)\mathrm{pr}(\sigma\mid\mathbf T)pr(σ∣T) the probability that RRR follows σ\sigmaσ, the expected cost is cˉR(T)=∑σc(T;σ) pr(σ∣T)\bar c_R(\mathbf T)=\sum_\sigma c(\mathbf T;\sigma)\,\mathrm{pr}(\sigma\mid\mathbf T)cˉR​(T)=∑σ​c(T;σ)pr(σ∣T). For w>0w>0w>0, RRR is expected www-competitive if there is a constant KKK with

cˉR(T)≤w c0(T)+K\bar c_R(\mathbf T)\le w\,c_0(\mathbf T)+KcˉR​(T)≤wc0​(T)+K

for every finite task sequence and every initial state. The randomized competitive ratio wˉ(S,d)\bar w(S,d)wˉ(S,d) is the infimum of all such www over all RRR.

Formalization targets

Goal: Theorem 7.1

For the uniform task system on n≥1n\ge1n≥1 states,

H(n)  ≤  wˉ(S,d)  ≤  2H(n).H(n)\;\le\;\bar w(S,d)\;\le\;2H(n).H(n)≤wˉ(S,d)≤2H(n).

Milestones

  1. Upper bound (p. 759). Some randomized on-line algorithm is expected 2H(n)2H(n)2H(n)-competitive on the uniform task system.
  2. Lemma 7.2 (p. 759). Let DDD be a distribution on infinite task sequences over a finite task alphabet, with E(c0(Tj))→∞E(c_0(\mathbf T^j))\to\inftyE(c0​(Tj))→∞, and let mj=inf⁡AE(cA(Tj))m_j=\inf_A E(c_A(\mathbf T^j))mj​=infA​E(cA​(Tj)) over deterministic on-line algorithms. Then every achievable www satisfies
lim sup⁡j→∞mjE(c0(Tj))≤w.\limsup_{j\to\infty}\frac{m_j}{E(c_0(\mathbf T^j))}\le w .j→∞limsup​E(c0​(Tj))mj​​≤w.
  1. mj≥j/nm_j\ge j/nmj​≥j/n (p. 760) when the tasks are independent uniformly random unit elementary tasks UsU_sUs​ (cost 111 in sss, 000 elsewhere).
  2. Coupon collector (p. 760). For i.i.d. uniform states on SSS, the expected number of draws until every state has appeared is nH(n)nH(n)nH(n).
  3. Off-line cost (p. 760). Under the same distribution, E(c0(Tj))≤j/(nH(n))+CE(c_0(\mathbf T^j))\le j/(nH(n))+CE(c0​(Tj))≤j/(nH(n))+C for a constant CCC independent of jjj.

Significance

The theorem shows that randomization reduces the competitive ratio of the uniform task system from 2n−12n-12n−1 to Θ(log⁡n)\Theta(\log n)Θ(logn). That is an exponential improvement, and it identifies the adversary's knowledge of the algorithm's state as the source of the deterministic lower bound. Lemma 7.2 is a form of Yao's principle adapted to the additive-constant definition of competitiveness. It is the standard tool for randomized lower bounds in on-line computation, and the same argument shape reappears for paging and kkk-server lower bounds.

On the formalization side, the result is proved but, as far as is known, has not been machine-checked. A complete development yields a reusable model of randomized on-line algorithms with oblivious adversaries, a Yao-type lemma usable for other on-line problems, and a coupon-collector expectation in the product-measure setting. The upper half additionally needs the continuous-time reduction of the paper's Lemma 3.1 in randomized form, or a direct discrete-time algorithm.

Difficulty

For the upper bound, the natural algorithm is continuous-time. It proceeds in phases, and inside a phase it stays in a state until that state has accumulated cost 111. A discrete task can saturate several states at once and straddle a phase boundary. So a discrete algorithm must either simulate the continuous one or be analyzed directly, and the expected transition count per phase must be controlled with the first phase starting in a deterministic state.

For the lower bound, the first obstacle is that the natural statement "wˉ≥lim sup⁡mj/E(c0)\bar w\ge\limsup m_j/E(c_0)wˉ≥limsupmj​/E(c0​)" silently assumes that a randomized algorithm's expected cost, averaged over random inputs, is at least that of the best deterministic algorithm. With the behavioural (kernel) definition used here, this requires converting a kernel into a mixture of deterministic algorithms, which is Kuhn's theorem on each finite horizon. The second obstacle is that the paper's claim E(c0(Tj))≤j/(nH(n))+O(1)E(c_0(\mathbf T^j))\le j/(nH(n))+O(1)E(c0​(Tj))≤j/(nH(n))+O(1) is supported only by the elementary renewal theorem, which gives a limit of ratios; the additive bound needs a sharper renewal estimate. Mathlib has no renewal theory. The hypothesis E(c0(Tj))→∞E(c_0(\mathbf T^j))\to\inftyE(c0​(Tj))→∞ of Lemma 7.2 must also be established for the uniform distribution; the paper does not prove it separately.

Formalization scope

  • States form a finite nonempty type S, and nnn = Fintype.card S; no n≥2n\ge2n≥2 assumption is made (at n=1n=1n=1 the goal reads 1≤wˉ≤21\le\bar w\le21≤wˉ≤2, and wˉ=1\bar w=1wˉ=1). H(n)H(n)H(n) is Mathlib's harmonic n, cast to R\mathbb RR. The uniform system has unit transition cost.
  • Tasks are finite and nonnegative. The paper also allows +∞+\infty+∞ entries; these are excluded. Task sequences are Fin m → S → ℝ and schedules are Fin (m+1) → S with σ 0 = s₀. c0c_0c0​ is a finite minimum.
  • A randomized algorithm is a kernel S → List (S → ℝ) → List S → PMF S. This is the paper's scheduler–taskmaster description (p. 758), equivalent to a distribution over deterministic algorithms on every finite task sequence. pr(σ∣T)\mathrm{pr}(\sigma\mid\mathbf T)pr(σ∣T) is the product of kernel probabilities, and cˉR\bar c_RcˉR​ is a finite sum. That pr(⋅∣T)\mathrm{pr}(\cdot\mid\mathbf T)pr(⋅∣T) sums to 111 has been checked locally.
  • wˉ(S,d)\bar w(S,d)wˉ(S,d) is the real sInf of {w:∃R, R expected w-competitive}\{w : \exists R,\ R \text{ expected } w\text{-competitive}\}{w:∃R, R expected w-competitive}. On the empty set this would be 000, so the upper bound is stated as the existence of an expected 2H(n)2H(n)2H(n)-competitive algorithm, and Lemma 7.2 is stated for every achievable www. The goal's lower half forces the set to be nonempty. Statements of the form "wˉ≤c\bar w\le cwˉ≤c" alone are therefore not acceptable substitutes for milestones 1 and 2.
  • Lemma 7.2 is restricted to task sequences over a finite alphabet, with the product σ\sigmaσ-algebra and measurable singletons. This makes every E(cA(Tj))E(c_A(\mathbf T^j))E(cA​(Tj)) a genuine integral for every deterministic AAA, and it covers the paper's application. The lim sup⁡\limsuplimsup of Lemma 7.2 is taken in EReal.
  • The coupon-collector time takes values in [0,∞][0,\infty][0,∞] and its expectation is a lower Lebesgue integral. Milestone 5 renders the paper's O(1)O(1)O(1) as an explicit constant CCC chosen before jjj.

Contributions welcome: proofs of any milestone; a discrete-time randomized phase algorithm; a general Kuhn-type conversion from kernels to mixtures of deterministic algorithms; renewal-theoretic lemmas.

Selected references

  • A. Borodin, N. Linial, M. Saks, An Optimal On-Line Algorithm for Metrical Task System, J. ACM 39(4):745–763, 1992. https://doi.org/10.1145/146585.146588
  • D. D. Sleator, R. E. Tarjan, Amortized Efficiency of List Update and Paging Rules, Commun. ACM 28(2):202–208, 1985. https://doi.org/10.1145/2786.2793
  • A. Fiat, R. M. Karp, M. Luby, L. A. McGeoch, D. D. Sleator, N. E. Young, Competitive Paging Algorithms, J. Algorithms 12(4):685–699, 1991. https://doi.org/10.1016/0196-6774(91)90041-V
  • A. C.-C. Yao, Probabilistic Computations: Toward a Unified Measure of Complexity, FOCS 1977, 222–227. https://doi.org/10.1109/SFCS.1977.24
  • S. M. Ross, Applied Probability Models with Optimization Applications, Holden-Day, 1970 (the elementary renewal theorem cited as [20] in the paper).
9 thms1 active userReviewed
🏆Completed
Machine Learning·Captain: Minghui

Neural Tangent Kernel: The Infinite-Width Initialization LimitResearch Paper

Why an initialization kernel matters

A neural network is nonlinear in its parameters, but a small change in those parameters changes its predictions through a Jacobian. The neural tangent kernel is the Gram kernel of that Jacobian: it records which changes in predictions can be produced by common parameter updates. An initialization limit identifies a deterministic object behind this random kernel. It supplies a mathematical starting point for studying wide networks through kernel methods, before addressing the additional question of how the kernel changes during training. Jacot, Gabriel, and Hongler establish this initialization limit in Section 4.1, Theorem 1, PDF p. 5.

The requested result is already a theorem of the paper. The open work here is its formal proof in Lean, including the probability model and the order of limits. The mission is classified as OpenProblem at the request of its proposer; that label does not assert that the underlying mathematical result remains an unsolved research question.

The source appeared in 2018 and was published at NeurIPS 2018; this formalization fixes arXiv version 4, dated February 10, 2020, so that page references and conventions remain stable. Its Appendix A explicitly distinguishes the sequential limit proved there from a possible stronger simultaneous-width limit.

Networks, randomness, and the two kernels

Fix positive integers ddd and qqq, the input and output dimensions, a hidden-layer count h≥0h\ge0h≥0, a bias scale β>0\beta>0β>0, and a Lipschitz function σ:R→R\sigma:\mathbb R\to\mathbb Rσ:R→R. Write L=h+1L=h+1L=h+1 for the number of affine layers, with n0=dn_0=dn0​=d, nL=qn_L=qnL​=q, and positive hidden widths n1,…,nhn_1,\ldots,n_hn1​,…,nh​. The parameters are all entries of the weight matrices and bias vectors. Every parameter is sampled independently from N(0,1)\mathcal N(0,1)N(0,1).

For an input xxx, let a(0)(x)=xa^{(0)}(x)=xa(0)(x)=x and define

zj(ℓ+1)(x)=1nℓ∑iWji(ℓ)ai(ℓ)(x)+βbj(ℓ).z^{(\ell+1)}_j(x)=\frac1{\sqrt{n_\ell}} \sum_i W^{(\ell)}_{ji}a^{(\ell)}_i(x)+\beta b^{(\ell)}_j.zj(ℓ+1)​(x)=nℓ​​1​i∑​Wji(ℓ)​ai(ℓ)​(x)+βbj(ℓ)​.

At each hidden layer, a(ℓ)=σ(z(ℓ))a^{(\ell)}=\sigma(z^{(\ell)})a(ℓ)=σ(z(ℓ)) coordinatewise. The output is fθ(x)=z(L)(x)f_\theta(x)=z^{(L)}(x)fθ​(x)=z(L)(x), with no final activation. These are the conventions of Section 2, PDF pp. 2–3. Matrix storage order in Lean uses destination then source; the displayed operation is unchanged.

For output coordinates k,k′k,k'k,k′, set

Θkk′(L)(θ;x,y)=∑p∂θpfθ,k(x) ∂θpfθ,k′(y).\Theta^{(L)}_{kk'}(\theta;x,y)=\sum_p \partial_{\theta_p}f_{\theta,k}(x)\, \partial_{\theta_p}f_{\theta,k'}(y).Θkk′(L)​(θ;x,y)=p∑​∂θp​​fθ,k​(x)∂θp​​fθ,k′​(y).

The sum includes every weight and every bias, as in Section 4, PDF p. 5.

The covariance kernel starts with

Σ(1)(x,y)=⟨x,y⟩d+β2.\Sigma^{(1)}(x,y)=\frac{\langle x,y\rangle}{d}+\beta^2.Σ(1)(x,y)=d⟨x,y⟩​+β2.

Given a centered Gaussian pair (U,V)(U,V)(U,V) with covariance matrix obtained by evaluating Σ(ℓ)\Sigma^{(\ell)}Σ(ℓ) on (x,y)(x,y)(x,y), define

Σ(ℓ+1)(x,y)=E[σ(U)σ(V)]+β2,Σ˙(ℓ+1)(x,y)=E[σ′(U)σ′(V)].\Sigma^{(\ell+1)}(x,y)=\mathbb E[\sigma(U)\sigma(V)]+\beta^2, \qquad \dot\Sigma^{(\ell+1)}(x,y)=\mathbb E[\sigma'(U)\sigma'(V)].Σ(ℓ+1)(x,y)=E[σ(U)σ(V)]+β2,Σ˙(ℓ+1)(x,y)=E[σ′(U)σ′(V)].

The deterministic limiting NTK is

Θ∞(1)=Σ(1),Θ∞(ℓ+1)(x,y)=Θ∞(ℓ)(x,y)Σ˙(ℓ+1)(x,y)+Σ(ℓ+1)(x,y).\Theta_\infty^{(1)}=\Sigma^{(1)},\qquad \Theta_\infty^{(\ell+1)}(x,y)=\Theta_\infty^{(\ell)}(x,y) \dot\Sigma^{(\ell+1)}(x,y)+\Sigma^{(\ell+1)}(x,y).Θ∞(1)​=Σ(1),Θ∞(ℓ+1)​(x,y)=Θ∞(ℓ)​(x,y)Σ˙(ℓ+1)(x,y)+Σ(ℓ+1)(x,y).

These recurrences appear in Section 4.1, Proposition 1 and Theorem 1, PDF p. 5; their displays are unnumbered.

Formalization targets

The goal is Theorem 1, on every fixed finite family X=(x1,…,xN)X=(x_1,\ldots,x_N)X=(x1​,…,xN​) of inputs and for every ε>0\varepsilon>0ε>0:

Pr⁡ ⁣(∃i,j,k,k′:∣Θkk′(L)(θ;xi,xj)−Θ∞(L)(xi,xj)δkk′∣>ε)⟶0.\Pr\!\left(\exists i,j,k,k': \left|\Theta^{(L)}_{kk'}(\theta;x_i,x_j) -\Theta_\infty^{(L)}(x_i,x_j)\delta_{kk'}\right|>\varepsilon\right) \longrightarrow0.Pr(∃i,j,k,k′:​Θkk′(L)​(θ;xi​,xj​)−Θ∞(L)​(xi​,xj​)δkk′​​>ε)⟶0.

Here δkk′\delta_{kk'}δkk′​ is one when the output coordinates coincide and zero otherwise. The limit takes n1→∞n_1\to\inftyn1​→∞ first, then n2→∞n_2\to\inftyn2​→∞, through nh→∞n_h\to\inftynh​→∞, precisely as specified in Appendix A, PDF p. 11, and Appendix A.1, PDF pp. 12–13.

Two supporting milestones expose the required mathematical content. First, the recursively defined Σ(L)\Sigma^{(L)}Σ(L) has positive semidefinite Gram matrices on every finite input family and satisfies Σ(L)(x,x)≥β2\Sigma^{(L)}(x,x)\ge\beta^2Σ(L)(x,x)≥β2. This is a paper-derived well-definedness obligation for the covariance in Proposition 1. Second, Proposition 1 asserts joint convergence in distribution of (fθ,k(xi))i,k(f_{\theta,k}(x_i))_{i,k}(fθ,k​(xi​))i,k​ to the centered Gaussian vector with covariance Σ(L)(xi,xj)δkk′\Sigma^{(L)}(x_i,x_j)\delta_{kk'}Σ(L)(xi​,xj​)δkk′​. This expresses the paper's independent output Gaussian processes through all their finite-dimensional distributions.

What completing the mission would establish

The result connects an explicitly parameterized random finite network to a deterministic kernel computed from its activation and depth. All output correlations, bias contributions, and layer normalizations remain visible in that connection. It would provide a checked foundation on which a separate training-stability development could build.

The formal contribution is the passage from finite random Jacobians to the limiting kernel. It does not assume that the empirical NTK already equals its limit. Nor does the goal claim convergence of a training trajectory, positive definiteness on a sphere, an early-stopping guarantee, or a generalization bound; those are separate results and questions in the source.

Where the mathematical work lies

The parameter space changes with the widths. Outputs, hidden activations, and their parameter derivatives are dependent random quantities, so the limit of their products requires more than a scalar law of large numbers. The weak Gaussian limit alone also does not justify substituting an arbitrary discontinuous derivative into expectations. The Lipschitz-only hypothesis is part of the target and must be retained, including for nonsmooth activations. Remark 3, PDF p. 5 identifies almost-everywhere differentiation as the relevant convention.

Formalization scope and conventions

Lean represents parameter coordinates by a finite dependent index carrying a layer, destination neuron, and optional source neuron; the missing source denotes a bias. Initialization is the finite product of Mathlib's standard Gaussian measures. Network outputs are obtained by the displayed recursion, and NTK entries use actual Fréchet derivatives in coordinate directions. Mathlib's derivative is zero where differentiation fails; the proof must establish that the exceptional parameters are null under the stated initialization law.

Centered Gaussian laws use Mathlib's multivariateGaussian, including singular covariance. The covariance-validity milestone must justify its covariance interpretation. No invertibility, distinct-input, smooth-activation, or positive-definite-kernel assumption is added. Positive actual hidden widths are indexed as wi+1w_i+1wi​+1 for wi∈Nw_i\in\mathbb Nwi​∈N, a cofinal reindexing. Depth one, constant activations, repeated or zero inputs, and empty finite families are included. The empty family is harmless because the same theorem quantifies over every nonempty family as well.

The sequential filter puts the last hidden width outermost. For two hidden widths, a probability tolerance is met by first choosing a threshold for n2n_2n2​, then a threshold for n1n_1n1​ that may depend on n2n_2n2​. There is no uniform limit over the entire input space and no simultaneous-width assertion. The development uses Lean 4.30.0 and supported Mathlib revision c5ea00351c28e24afc9f0f84379aa41082b1188f. The model and statements are locally checked; the three theorem proofs remain open. Contributions to Gaussian covariance consistency, finite-dimensional distribution limits, almost-everywhere network differentiation, and the NTK limit are in scope.

Selected references

  • Arthur Jacot, Franck Gabriel, and Clément Hongler, Neural Tangent Kernel: Convergence and Generalization in Neural Networks, Advances in Neural Information Processing Systems 31, 2018. arXiv:1806.07572v4. Primary anchors: Section 2, PDF pp. 2–3; Section 4.1, PDF p. 5, Proposition 1, Theorem 1, Remarks 2–3; Appendix A and A.1, PDF pp. 11–13. Relevant displays have no equation numbers.
4 thms1 active userReviewed
🏆Completed
Machine LearningOptimization·Captain: Minghui

Optimization Methods for Large-Scale Machine Learning: Stochastic Gradient ConvergenceResearch Paper

Why stochastic-gradient convergence matters

Training a machine-learning model often means choosing a vector of parameters to minimize an average loss. Evaluating the full gradient can require processing an entire dataset. A stochastic-gradient method instead updates the parameters using a random direction obtained from a smaller amount of information. Its computational appeal raises a mathematical question: which assumptions on those directions and the stepsizes guarantee progress, and what kind of convergence follows?

This mission formalizes the core convergence theory in Section 4 of Bottou, Curtis, and Nocedal, Optimization Methods for Large-Scale Machine Learning. The results distinguish strongly convex objectives, where expected objective error can be controlled, from general smooth objectives, where the guarantee concerns gradients. They also distinguish constant stepsizes, which leave a noise-dependent error bound, from diminishing stepsizes.

Historical timeline

  • 1951: Robbins and Monro introduced stochastic approximation for finding a root using noisy observations. Their work is the historical foundation for the stepsize conditions used here; the original root-finding theorem is not a separate target of this mission. Original paper.
  • 2016: Bottou, Curtis, and Nocedal released the first version of their survey, organizing stochastic-gradient theory around smoothness and moment assumptions. arXiv record.
  • 2018: The revised survey appeared in SIAM Review. This mission fixes arXiv version 3 for stable theorem numbering and PDF page citations. Published article.

The mathematics targeted here is already proved in the literature; the task is its Lean formalization, not a claim that these convergence results are unresolved research conjectures.

Objective, algorithm, and probability model

Let F:Rd→RF:\mathbb R^d\to\mathbb RF:Rd→R be differentiable with an LLL-Lipschitz gradient, where L>0L>0L>0. On a probability space (Ω,A,P)(\Omega,\mathcal A,\mathbb P)(Ω,A,P), let Hk\mathcal H_kHk​ contain the history before step kkk. Starting from a deterministic vector w0w_0w0​, the algorithm uses positive deterministic stepsizes αk\alpha_kαk​ and random directions gkg_kgk​ to update

wk+1=wk−αkgk.w_{k+1}=w_k-\alpha_k g_k.wk+1​=wk​−αk​gk​.

The state wkw_kwk​ is measurable with respect to Hk\mathcal H_kHk​; the direction gkg_kgk​ is measurable with respect to Hk+1\mathcal H_{k+1}Hk+1​ and has a finite second moment. Conditional assertions hold almost surely. This uses the adapted-process formulation expressly permitted in footnote 4, PDF p. 22, rather than requiring independent sample seeds. The local index k=0k=0k=0 corresponds to the paper's k=1k=1k=1. Algorithm 4.1 and footnote 4.

Write Ek\mathbb E_kEk​ for conditioning on Hk\mathcal H_kHk​. The moment conditions use constants μG≥μ>0\mu_G\ge\mu>0μG​≥μ>0 and M,MV≥0M,M_V\ge0M,MV​≥0:

⟨∇F(wk),Ekgk⟩≥μ∥∇F(wk)∥2,∥Ekgk∥≤μG∥∇F(wk)∥,\langle\nabla F(w_k),\mathbb E_k g_k\rangle\ge\mu\|\nabla F(w_k)\|^2, \qquad \|\mathbb E_k g_k\|\le\mu_G\|\nabla F(w_k)\|,⟨∇F(wk​),Ek​gk​⟩≥μ∥∇F(wk​)∥2,∥Ek​gk​∥≤μG​∥∇F(wk​)∥, Ek∥gk∥2−∥Ekgk∥2≤M+MV∥∇F(wk)∥2.\mathbb E_k\|g_k\|^2-\|\mathbb E_k g_k\|^2 \le M+M_V\|\nabla F(w_k)\|^2.Ek​∥gk​∥2−∥Ek​gk​∥2≤M+MV​∥∇F(wk​)∥2.

Define MG=MV+μG2M_G=M_V+\mu_G^2MG​=MV​+μG2​. The iterates lie almost surely in an open region on which F≥Finf⁡F\ge F_{\inf}F≥Finf​ for a real lower bound Finf⁡F_{\inf}Finf​. These are Assumptions 4.1 and 4.3, PDF pp. 23–24, equations (4.6)–(4.9). Source.

Formalization targets

The goal is Theorem 4.10, Section 4.3, PDF p. 33, equations (4.30a)–(4.30b). Suppose

∑k=0∞αk=∞,∑k=0∞αk2<∞.\sum_{k=0}^\infty\alpha_k=\infty,\qquad \sum_{k=0}^\infty\alpha_k^2<\infty.k=0∑∞​αk​=∞,k=0∑∞​αk2​<∞.

For AK=∑k=0K−1αkA_K=\sum_{k=0}^{K-1}\alpha_kAK​=∑k=0K−1​αk​, establish both

∃S∈R:E ⁣[∑k=0K−1αk∥∇F(wk)∥2]⟶S,\exists S\in\mathbb R:\quad \mathbb E\!\left[\sum_{k=0}^{K-1}\alpha_k\|\nabla F(w_k)\|^2\right]\longrightarrow S,∃S∈R:E[k=0∑K−1​αk​∥∇F(wk​)∥2]⟶S, 1AKE ⁣[∑k=0K−1αk∥∇F(wk)∥2]⟶0.\frac{1}{A_K}\mathbb E\!\left[\sum_{k=0}^{K-1}\alpha_k\|\nabla F(w_k)\|^2\right] \longrightarrow0.AK​1​E[k=0∑K−1​αk​∥∇F(wk​)∥2]⟶0.

There is no convexity assumption and no restriction that the initial stepsizes already satisfy a small-step bound. Theorem 4.10.

Four milestones capture the surrounding theory. Lemma 4.4 gives the two successive conditional expected-descent inequalities (4.10a)–(4.10b), PDF pp. 24–25. It is a common input to the convergence results. Theorem 4.6 gives a geometric upper bound for strongly convex objectives with a constant stepsize. Theorem 4.7 gives the corresponding O(1/k)O(1/k)O(1/k) bound for αk=β/(γ+k+1)\alpha_k=\beta/(\gamma+k+1)αk​=β/(γ+k+1). These are parallel strongly convex targets, not prerequisites for the nonconvex goal. Theorem 4.8 gives finite-horizon sum and average squared-gradient bounds for general objectives with a constant stepsize. Section 4.

For example, if c>0c>0c>0 is the strong-convexity constant, d≥1d\ge1d≥1, and 0<a≤μ/(LMG)0<a\le\mu/(LM_G)0<a≤μ/(LMG​), Theorem 4.6 states, with B=aLM/(2cμ)B=aLM/(2c\mu)B=aLM/(2cμ) and F∗=inf⁡xF(x)F_*=\inf_xF(x)F∗​=infx​F(x),

E[F(wk)−F∗]≤B+(1−acμ)k(F(w0)−F∗−B).\mathbb E[F(w_k)-F_*]\le B+(1-ac\mu)^k(F(w_0)-F_*-B).E[F(wk​)−F∗​]≤B+(1−acμ)k(F(w0​)−F∗​−B).

The upper bound tends to BBB; this does not assert that the actual expected error tends to BBB. Every milestone retains the paper's constants and all displayed conclusions. Equations (4.13)–(4.14), PDF p. 26.

What a completed formalization provides

The result explains precisely how noise and stepsize interact. The general-objective goal guarantees that the stepsize-weighted expected squared gradients average to zero even when M>0M>0M>0. The strongly convex milestones quantify objective error and the effect of initialization. These results concern the stated quantities; they do not assert convergence of iterates, global optimality for a nonconvex objective, or almost-sure convergence. Sections 4.2–4.3.

A completed development would provide reusable Lean results for smooth objective functions, conditional moment bounds, stochastic updates, and expected convergence. The proposed statements are open proof obligations. Compilation establishes that the definitions and statements are well formed, not that their convergence claims have already been proved.

Mathematical and formal difficulties

Finite conditional expectations must be connected to unconditional integrals without relying on total-function defaults. The infinite-horizon result also requires careful handling of a finite initial segment: square summability gives eventually small steps, not a bound at every step. Strong convexity must justify the objective-gap estimates and the properties of the optimum. Treating a descent recurrence as a hypothesis would omit the analytic content that this mission is intended to formalize.

Formalization scope

The model uses real finite-dimensional Euclidean space, Mathlib gradients, filtrations, Bochner conditional expectations, and ordinary real integrals. The probability space is arbitrary; no finite-support or standard-Borel restriction is imposed. Directions have explicit finite second moments, making the finite-expectation convention in the paper visible. Square integrability of iterates and integrability of losses are consequences to establish, not extra model fields.

Strong convexity uses Mathlib's StrongConvexOn, equivalent here to Assumption 4.5's first-order inequality. The optimum is defined as the infimum of the range of FFF, with its finiteness to be derived in the strongly convex branch. That branch requires d≥1d\ge1d≥1: the paper's deduction c≤Lc\le Lc≤L implicitly uses a nontrivial space. The nonconvex statements allow d=0d=0d=0. Zero noise is allowed. Finite-horizon averages require K>0K>0K>0; the value assigned at K=0K=0K=0 does not affect an asymptotic limit.

Contributions to the conditional-descent infrastructure and any of the four milestones are welcome. Variance reduction, Newton-type methods, and the remainder of the survey are outside this initial mission.

Selected references

  • Léon Bottou, Frank E. Curtis, and Jorge Nocedal. Optimization Methods for Large-Scale Machine Learning. SIAM Review 60(2), 223–311, 2018. arXiv:1606.04838v3; DOI. All page numbers above refer to the 95-page arXiv v3 PDF.
  • Herbert Robbins and Sutton Monro. A Stochastic Approximation Method. Annals of Mathematical Statistics 22(3), 400–407, 1951. DOI. Historical background only.
6 thms1 active userReviewed
🏆Completed
Numerical Analysis·Captain: shivm

Randomized Kaczmarz: Exponential Convergence in ExpectationResearch Paper

Problem

Solve a consistent system Ax=bAx=bAx=b, with A∈Cm×nA\in\mathbb{C}^{m\times n}A∈Cm×n of full rank and m≥n≥1m\ge n\ge 1m≥n≥1. Kaczmarz's method (1937) projects the current iterate onto the solution hyperplane of one equation at a time. The cyclic version converges, but its rate depends on the order of the rows and has no clean bound in terms of a condition number.

Strohmer and Vershynin (2009) pick row iii at random with probability ∥ai∥22/∥A∥F2\|a_i\|_2^2/\|A\|_F^2∥ai​∥22​/∥A∥F2​ and prove

E ∥xk−x∥22≤(1−κ(A)−2)k ∥x0−x∥22,κ(A)=∥A∥Fσmin⁡(A).\mathbb{E}\,\|x_k-x\|_2^2 \le \bigl(1-\kappa(A)^{-2}\bigr)^k\,\|x_0-x\|_2^2,\qquad \kappa(A)=\frac{\|A\|_F}{\sigma_{\min}(A)}.E∥xk​−x∥22​≤(1−κ(A)−2)k∥x0​−x∥22​,κ(A)=σmin​(A)∥A∥F​​.

The rate does not depend on the number of equations mmm.

Setting

  • Rows. ai∈Cna_i\in\mathbb{C}^nai​∈Cn is the conjugate of row iii, so equation iii reads ⟨ai,x⟩=bi\langle a_i,x\rangle=b_i⟨ai​,x⟩=bi​.
  • Condition number. σmin⁡(A)=inf⁡∥z∥2=1∥Az∥2\sigma_{\min}(A)=\inf_{\|z\|_2=1}\|Az\|_2σmin​(A)=inf∥z∥2​=1​∥Az∥2​ and κ(A)=∥A∥F/σmin⁡(A)\kappa(A)=\|A\|_F/\sigma_{\min}(A)κ(A)=∥A∥F​/σmin​(A) (Demmel). It satisfies n≤κ(A)≤n ∥A∥2/σmin⁡(A)\sqrt n\le\kappa(A)\le\sqrt n\,\|A\|_2/\sigma_{\min}(A)n​≤κ(A)≤n​∥A∥2​/σmin​(A).
  • Algorithm 1. From any x0x_0x0​, draw row rrr independently with probability pr=∥ar∥22/∥A∥F2p_r=\|a_r\|_2^2/\|A\|_F^2pr​=∥ar​∥22​/∥A∥F2​ and set
xk+1=xk+br−⟨ar,xk⟩∥ar∥22 ar.x_{k+1}=x_k+\frac{b_r-\langle a_r,x_k\rangle}{\|a_r\|_2^2}\,a_r .xk+1​=xk​+∥ar​∥22​br​−⟨ar​,xk​⟩​ar​.
  • Expectation. E∥xk−x∥22\mathbb{E}\|x_k-x\|_2^2E∥xk​−x∥22​ is a finite sum over the mkm^kmk possible row sequences.

Targets

  • Theorem 2 (goal): the bound above, for every x0x_0x0​ and kkk.
  • Theorem 3: some x0≠xx_0\ne xx0​=x has E∥xk−x∥22≥(1−2k/κ(A)2)∥x0−x∥22\mathbb{E}\|x_k-x\|_2^2\ge(1-2k/\kappa(A)^2)\|x_0-x\|_2^2E∥xk​−x∥22​≥(1−2k/κ(A)2)∥x0​−x∥22​ for all k≥1k\ge1k≥1, so κ(A)−2\kappa(A)^{-2}κ(A)−2 is sharp up to a constant. Caveat: the source's proof only reaches the unsquared bound E∥xk−x∥2≥…\mathbb{E}\|x_k-x\|_2\ge\dotsE∥xk​−x∥2​≥… and then cites Jensen, which goes the wrong way. Its estimates give the squared bound with 444 in place of 222. Both forms are milestones.
  • Sharpness (§3.2): Theorem 2 is an equality when κ(A)=n\kappa(A)=\sqrt nκ(A)=n​.
  • Iteration count (§2.1): k≥2log⁡ε/log⁡(1−κ(A)−2)k\ge 2\log\varepsilon/\log(1-\kappa(A)^{-2})k≥2logε/log(1−κ(A)−2) steps give E∥xk−x∥22≤ε2∥x0−x∥22\mathbb{E}\|x_k-x\|_2^2\le\varepsilon^2\|x_0-x\|_2^2E∥xk​−x∥22​≤ε2∥x0​−x∥22​.

Proof idea

A single projection can barely reduce the error, when the error is almost orthogonal to the chosen row. On average it always does. If Z=aj/∥aj∥2Z=a_j/\|a_j\|_2Z=aj​/∥aj​∥2​ is drawn with probability pjp_jpj​, then

E ∣⟨z,Z⟩∣2=∥Az∥22∥A∥F2≥κ(A)−2∥z∥22.\mathbb{E}\,|\langle z,Z\rangle|^2=\frac{\|Az\|_2^2}{\|A\|_F^2}\ge\kappa(A)^{-2}\|z\|_2^2 .E∣⟨z,Z⟩∣2=∥A∥F2​∥Az∥22​​≥κ(A)−2∥z∥22​.

Pythagoras for one projection then gives E∥xk+1−x∥22≤(1−κ(A)−2) ∥xk−x∥22\mathbb{E}\|x_{k+1}-x\|_2^2\le(1-\kappa(A)^{-2})\,\|x_k-x\|_2^2E∥xk+1​−x∥22​≤(1−κ(A)−2)∥xk​−x∥22​; induct on kkk.

Formalization

  • Vectors live in EuclideanSpace ℂ (Fin n) and matrices in Matrix (Fin m) (Fin n) ℂ. Mathlib's inner product is conjugate-linear in its first argument, which is why aia_iai​ is a conjugated row.
  • Full rank is stated as injectivity of z↦Azz\mapsto Azz↦Az.
  • The expectation expErrSq is an explicit finite sum, so no measure theory is needed. The tower identity is a milestone.
  • A zero row makes the step the identity (Lean's division by zero) and has probability 000, so it is harmless.
  • Mathlib has no Kaczmarz iteration, scaled condition number or σmin⁡\sigma_{\min}σmin​ lower bound; the mission builds them.

History

  • 1937: Kaczmarz introduces the cyclic method and proves convergence, with no rate.
  • 1970: Gordon, Bender and Herman rediscover it as ART for tomography.
  • 2009: Strohmer and Vershynin give the first rate in terms of a condition number.
  • 2010 onward: extensions to noisy systems, block methods, SGD and sketch-and-project (Needell; Needell–Tropp; Needell–Srebro–Ward; Gower–Richtárik).

References

  • S. Kaczmarz, Angenäherte Auflösung von Systemen linearer Gleichungen, Bull. Int. Acad. Polon. Sci. Lett. A 35 (1937), 355–357.
  • T. Strohmer and R. Vershynin, A randomized Kaczmarz algorithm with exponential convergence, J. Fourier Anal. Appl. 15 (2009), 262–278. arXiv:math/0702226
  • J. Demmel, The probability that a numerical analysis problem is difficult, Math. Comp. 50 (1988), 449–480. DOI
  • D. Needell, Randomized Kaczmarz solver for noisy linear systems, BIT Numer. Math. 50 (2010), 395–403. arXiv:0902.0958
  • D. Needell, N. Srebro and R. Ward, Stochastic gradient descent, weighted sampling, and the randomized Kaczmarz algorithm, Math. Program. 155 (2016), 549–573. arXiv:1310.5715
  • R. M. Gower and P. Richtárik, Randomized iterative methods for linear systems, SIAM J. Matrix Anal. Appl. 36 (2015), 1660–1690. arXiv:1506.03296
12 thms1 active userReviewed
🏆Completed
Active InferenceBehaviorDynamical Systems+4·Captain: ActiveInference

Free Energy Principle I: the variational free-energy boundResearch Paper

Motivation

The free energy principle (FEP) proposes that a self-organizing system — a brain, an organism, an agent — persists by minimizing one quantity: the variational free energy of its sensory states under an internal generative model. Introduced by Karl Friston as a principle of brain function [Friston 2006] and stated in its unified form [Friston 2010], the principle makes a precise mathematical claim at its core: whatever internal state estimate the system holds, the free energy of incoming data is never below the data's surprisal (negative log marginal likelihood), and the excess is exactly the Kullback–Leibler divergence between the system's recognition density and the Bayesian posterior implied by the model. Active inference extends the same functional from perception to action and planning [Friston et al. 2017], and the same bound is known in machine learning as the evidence lower bound (ELBO) of variational inference [Parr et al. 2022].

Timeline of the mathematical content this mission formalizes:

  • 2006 — Friston, A free energy principle for the brain (J. Physiol. Paris 100): the bound stated for perception as variational inference on a generative model.
  • 2010 — Friston, The free-energy principle: a unified brain theory? (Nat. Rev. Neurosci. 11, 127–138): free energy as an upper bound on surprisal, presented as the core of a unified account.
  • 2017 — Friston, FitzGerald, Rigoli, Schwartenbeck, Pezzulo, Active inference: a process theory (Neural Comput. 29(1), 1–49): the same functional drives policy selection through expected free energy.
  • 2022 — Parr, Pezzulo, Friston, Active Inference (MIT Press): textbook treatment; the posterior-form identity F=DKL(Q ∥ P(s∣o))−log⁡P(o)F = D_{\mathrm{KL}}(Q\,\|\,P(s|o)) - \log P(o)F=DKL​(Q∥P(s∣o))−logP(o) as the central equation.
  • 2026 — fep_formal (Active Inference Institute): a machine-checked Lean 4 catalogue of 155 Free Energy Principle topics, compiled with zero proof holes against a pinned Mathlib. This mission transcribes the catalogue's core-free-energy chain — topic fep-002 and the foundation module active_inference — onto the platform, turning the first link of the FEP development into solvable community infrastructure.

Setting

Everything is finite, and laws are normalized real mass functions.

A finite law on a finite type α\alphaα is a function p:α→Rp : \alpha \to \mathbb{R}p:α→R with p(x)≥0p(x) \ge 0p(x)≥0 for every xxx and ∑xp(x)=1\sum_x p(x) = 1∑x​p(x)=1. A finite kernel from α\alphaα to β\betaβ assigns to each x∈αx \in \alphax∈α a normalized row over β\betaβ. The mission's definitions Def_fep_finite_laws and Def_fep_finite_information package these carriers with entropy, cross-entropy, and the KL divergence

DKL(p ∥ q)  =  ∑xq(x)⋅klFun ⁣(p(x)q(x)),klFun(x)=xlog⁡x+1−x,D_{\mathrm{KL}}(p\,\|\,q) \;=\; \sum_{x} q(x)\cdot \mathrm{klFun}\!\left(\frac{p(x)}{q(x)}\right), \qquad \mathrm{klFun}(x) = x\log x + 1 - x,DKL​(p∥q)=x∑​q(x)⋅klFun(q(x)p(x)​),klFun(x)=xlogx+1−x,

a totalized real-valued divergence that is finite even at zero-mass atoms (the convention 0⋅log⁡0=00 \cdot \log 0 = 00⋅log0=0 via Real.negMulLog) and nonnegative on normalized laws.

A finite generative model for active inference (definition Def_fep_generative_model) over finite types Policy,State,Outcome\mathsf{Policy}, \mathsf{State}, \mathsf{Outcome}Policy,State,Outcome consists of: an initial state law P(s)P(s)P(s); a policy-conditioned transition kernel; a state-to-outcome likelihood kernel; a preference law over outcomes; and a policy prior. Under a policy π\piπ the model predicts the state law P(s∣π)P(s \mid \pi)P(s∣π) and the outcome law P(o∣π)P(o \mid \pi)P(o∣π). A recognition density is any finite law QQQ over states — the system's internal estimate. At an outcome ooo with positive predicted mass, the Bayesian posterior P(⋅∣o,π)P(\cdot \mid o, \pi)P(⋅∣o,π) is the exact finite Bayes rule. The outcome surprisal is −log⁡P(o∣π)-\log P(o \mid \pi)−logP(o∣π), and the posterior-form variational free energy of a recognition density QQQ is

F[Q,o,π]  =  DKL(Q ∥ P(⋅∣o,π))  −  log⁡P(o∣π).F[Q, o, \pi] \;=\; D_{\mathrm{KL}}\big(Q \,\|\, P(\cdot \mid o, \pi)\big) \;-\; \log P(o \mid \pi).F[Q,o,π]=DKL​(Q∥P(⋅∣o,π))−logP(o∣π).

Formalization targets

Goal: the variational free-energy bound

For every generative model, every policy π\piπ, every outcome ooo with P(o∣π)>0P(o\mid\pi) > 0P(o∣π)>0, and every recognition density QQQ:

−log⁡P(o∣π)  ≤  F[Q,o,π].-\log P(o \mid \pi) \;\le\; F[Q, o, \pi].−logP(o∣π)≤F[Q,o,π].

The recognition density QQQ is universally quantified — the bound holds for whatever state estimate the system happens to carry.

Exactness, uniqueness, and the ELBO form

Three companions pin down the equality case, ordered weakest to strongest alongside the milestone list:

  • Exactness — the Bayesian posterior attains the bound:
F[P(⋅∣o,π), o, π]=−log⁡P(o∣π).F\big[P(\cdot \mid o, \pi),\, o,\, \pi\big] = -\log P(o \mid \pi).F[P(⋅∣o,π),o,π]=−logP(o∣π).
  • Uniqueness — equality characterizes the posterior, with no full-support assumption:
F[Q,o,π]=−log⁡P(o∣π)  ⟺  Q=P(⋅∣o,π).F[Q, o, \pi] = -\log P(o \mid \pi) \iff Q = P(\cdot \mid o, \pi).F[Q,o,π]=−logP(o∣π)⟺Q=P(⋅∣o,π).
  • ELBO form — negating both sides:
−F[Q,o,π]  ≤  log⁡P(o∣π).-F[Q, o, \pi] \;\le\; \log P(o \mid \pi).−F[Q,o,π]≤logP(o∣π).

Measure-theoretic core

Independently of the finite model, in Mathlib's nonnegative extended reals R≥0∪{∞}\mathbb{R}_{\ge0} \cup \{\infty\}R≥0​∪{∞}, with qqq, ppp measures on any measurable space and s∈R≥0∪{∞}s \in \mathbb{R}_{\ge0} \cup \{\infty\}s∈R≥0​∪{∞}:

s  ≤  s+DKL(q ∥ p),s \;\le\; s + D_{\mathrm{KL}}(q \,\|\, p),s≤s+DKL​(q∥p),

the unconditional shape of the bound (topic fep-002 of the source catalogue), with the divergence taken as ∞\infty∞ when the log-likelihood ratio is not integrable.

Significance

The result itself. This inequality is the load-bearing step of the FEP: it converts "minimize free energy" into "move recognition toward the posterior," and it is the exact statement whose continuous, dynamic, and policy-selecting extensions (expected free energy, Markov blankets, non-equilibrium thermodynamics) form the rest of the FEP literature. Without it, the principle's variational step has no mathematical content.

Formalizing it. The mathematics here is classical — Gibbs' inequality — and the source development already proves every row with no proof holes. What the mission adds is faithful, reusable infrastructure: the definitions are published as platform nodes in the shared namespace FreeEnergyPrinciple, so later missions in this programme (expected free energy and policy selection, Markov blankets, Gaussian and continuous-time variants, already proved in the source repository) can import them instead of re-deriving the substrate. Status honesty: all eight items below are formalized and machine-checked locally against the platform environment; each is an open problem on the platform only in the sense that no proof has yet been submitted to it.

Difficulty

The bound itself is a one-line consequence of KL nonnegativity — the naive idea "prove it by simp on the KL sum" is essentially right, and the source proofs are correspondingly short. The actual difficulty is boundary precision, where plausible renderings go silently wrong:

  • The positivity premise P(o∣π)>0P(o \mid \pi) > 0P(o∣π)>0 is not decoration: the Bayesian posterior is defined only where the evidence has positive mass, and hiding that in a totalized division would change the statement.
  • The uniqueness characterization is not a formality: at zero-mass reference atoms the logarithmic cross-entropy identity degenerates, and the proof needs the normalization lemma DKL(p ∥ q)=0↔p=qD_{\mathrm{KL}}(p\,\|\,q) = 0 \leftrightarrow p = qDKL​(p∥q)=0↔p=q, which forces the recognition law's mass to zero wherever the posterior's is zero. A solver who proves the bound but states equality with a full-support hypothesis has proved something different from the source.
  • The measure-theoretic core is deliberately unconditional; adding finiteness side conditions to it would weaken the source's point that R≥0∪{∞}\mathbb{R}_{\ge0} \cup \{\infty\}R≥0​∪{∞} absorbs the degenerate cases.

A vacuous formalization — quantifying over a single distinguished recognition law, or taking "posterior" as an arbitrary variable — would trivialize the goal; the targets below rule this out by fixing the exact finite Bayes rule and universally quantifying QQQ.

Formalization scope

Committed conventions of this mission's Lean development:

  • All model carriers are finite types (Fintype); laws are R\mathbb{R}R-valued normalized mass functions; kernels are normalized rows. No measure-theoretic machinery below the finite substrate except for the measure-theoretic core milestone.
  • KL is the totalized real-valued finite divergence DKL(p ∥ q)=∑xq(x)⋅klFun(p(x)/q(x))D_{\mathrm{KL}}(p\,\|\,q) = \sum_x q(x)\cdot \mathrm{klFun}(p(x)/q(x))DKL​(p∥q)=∑x​q(x)⋅klFun(p(x)/q(x)); entropy uses Real.negMulLog, so 0⋅log⁡0=00 \cdot \log 0 = 00⋅log0=0 exactly, not by exception-handling.
  • The posterior is the exact finite Bayes rule FiniteKernel.posterior, taken at the explicit hypothesis 0<P(o∣π)0 < P(o \mid \pi)0<P(o∣π).
  • One mission-wide namespace FreeEnergyPrinciple; definitions live in the published definition files Def_fep_finite_laws, Def_fep_finite_information, Def_fep_generative_model, and every theorem item imports them. A Free Energy Principle II mission (expected free energy) is expected to reuse the same namespace and definitions.
  • The measure-theoretic core uses Mathlib's InformationTheory.klDiv in ℝ≥0∞ with no finiteness hypotheses.
  • Contributions welcome: alternative measure-theoretic renderings of the core bound, the Gaussian instantiation of the same identity, and ports of the source repository's subsequent rows (Bayesian model reduction, expected free energy) onto these definitions.

Selected references

  • K. Friston, A free energy principle for the brain, Journal of Physiology (Paris) 100 (2006) 70–87. https://doi.org/10.1016/j.jphysparis.2006.10.001
  • K. Friston, The free-energy principle: a unified brain theory?, Nature Reviews Neuroscience 11 (2010) 127–138. https://doi.org/10.1038/nrn2787
  • K. Friston, T. FitzGerald, F. Rigoli, P. Schwartenbeck, G. Pezzulo, Active inference: a process theory, Neural Computation 29 (2017) 1–49. https://doi.org/10.1162/neco_a_00912
  • T. Parr, G. Pezzulo, K. J. Friston, Active Inference: The Free Energy Principle in Mind, Brain, and Behavior, MIT Press (2022). https://mitpress.mit.edu/9780262045354/active-inference/
  • D. A. Friedman, fep_formal: Towards Lean 4 Formalization of the Free Energy Principle (v1.2.0), Active Inference Institute (2026), the formal source of truth for this mission. https://github.com/ActiveInferenceInstitute/fep_formal
  • D. A. Friedman, Towards Lean 4 Formalization of the Free Energy Principle: AI-Driven Theorem Sketching and Verification for Active Inference and Bayesian Mechanics, Active Inference Journal (2026). https://doi.org/10.5281/zenodo.19699233
8 thms1 active userReviewed
🏆Completed
Machine LearningOptimization·Captain: Minghui

SCAFFOLD: Convergence with Client SamplingResearch Paper

Why control variates matter in federated optimization

Federated optimization trains one model using objectives held by many clients. Performing several local updates between communication rounds saves communication, but clients with different objectives can move in different directions. SCAFFOLD maintains a correction for each client to address this disagreement, including when only some clients participate. Karimireddy et al. establish convergence without a bound on how similar the client objectives are. The relevant result is Theorem III, Section 5, PDF p. 5.

SCAFFOLD: Convergence with Client Sampling concerns that known result's formalization. Its exact targets are finite-round bounds from the Appendix E analysis, covering strongly convex, general convex, and nonconvex objectives. The paper's FedAvg results and its separate quadratic-acceleration theorem are outside this scope.

Setting: local updates and sampled clients

There are N≥1N\ge1N≥1 clients, with differentiable objectives fi:Rd→Rf_i:\mathbb R^d\to\mathbb Rfi​:Rd→R. The global objective is f(x)=N−1∑ifi(x)f(x)=N^{-1}\sum_i f_i(x)f(x)=N−1∑i​fi​(x). Every client gradient is β\betaβ-Lipschitz, where β>0\beta>0β>0. A stochastic gradient query is conditionally unbiased and has conditional squared-error expectation at most σ2\sigma^2σ2, with σ≥0\sigma\ge0σ≥0. Current queries on different clients are conditionally independent given the full history. These make precise the fresh-oracle interpretation of A4–A5, Appendix B.1, PDF p. 14.

Each of T≥1T\ge1T≥1 rounds selects a uniform subset ArA_rAr​ of SSS clients, where 1≤S≤N1\le S\le N1≤S≤N. Each selected client takes K≥1K\ge1K≥1 local steps. Let xrx^rxr be the server model, circ_i^rcir​ its stored client control variates, and cr=N−1∑icirc^r=N^{-1}\sum_i c_i^rcr=N−1∑i​cir​. With constant local and global step sizes ηl>0\eta_l>0ηl​>0 and ηg≥1\eta_g\ge1ηg​≥1, a client starts at yi,0r=xry_{i,0}^r=x^ryi,0r​=xr and uses

yi,k+1r=yi,kr−ηl(gi,kr−cir+cr),xr+1=xr+ηgS∑i∈Ar(yi,Kr−xr).y_{i,k+1}^r=y_{i,k}^r-\eta_l\bigl(g_{i,k}^r-c_i^r+c^r\bigr), \qquad x^{r+1}=x^r+\frac{\eta_g}{S}\sum_{i\in A_r}(y_{i,K}^r-x^r).yi,k+1r​=yi,kr​−ηl​(gi,kr​−cir​+cr),xr+1=xr+Sηg​​i∈Ar​∑​(yi,Kr​−xr).

Option II replaces a participating client's control with K−1∑k=0K−1gi,krK^{-1}\sum_{k=0}^{K-1}g_{i,k}^rK−1∑k=0K−1​gi,kr​ and retains every other control. These are Algorithm 1, PDF p. 4, and Appendix E, equations (18)–(21), PDF pp. 25–26. Write h=Kηlηgh=K\eta_l\eta_gh=Kηl​ηg​ for the effective step size.

Formalization targets

Four milestones support a root goal that is the conjunction of the two convergence statements below.

GradientGrowth states, for convex clients and a global minimizer x⋆x^\starx⋆,

1N∑i∥∇fi(x)−∇fi(x⋆)∥2≤2β(f(x)−f(x⋆)).\frac1N\sum_i\|\nabla f_i(x)-\nabla f_i(x^\star)\|^2 \le2\beta\bigl(f(x)-f(x^\star)\bigr).N1​i∑​∥∇fi​(x)−∇fi​(x⋆)∥2≤2β(f(x)−f(x⋆)).

This is Appendix B.1, equation (9), PDF p. 14. PerturbedStrongConvexity states, for each μ\muμ-strongly convex client and μ≥0\mu\ge0μ≥0,

⟨∇fi(x),z−y⟩≥fi(z)−fi(y)+μ4∥y−z∥2−β∥z−x∥2,\langle\nabla f_i(x),z-y\rangle\ge f_i(z)-f_i(y) +\frac\mu4\|y-z\|^2-\beta\|z-x\|^2,⟨∇fi​(x),z−y⟩≥fi​(z)−fi​(y)+4μ​∥y−z∥2−β∥z−x∥2,

as in Appendix C, Lemma 5, PDF p. 17.

ConvexFiniteRoundConvergence allows arbitrary deterministic initial controls ci0c_i^0ci0​. Assume all clients are μ\muμ-strongly convex with μ≥0\mu\ge0μ≥0, including ordinary convexity when μ=0\mu=0μ=0, and fix a minimizer x⋆x^\starx⋆ of fff. Define

C0=1N∑i∥ci0−∇fi(x⋆)∥2,V0=∥x0−x⋆∥2+9Nh2SC0,C_0=\frac1N\sum_i\|c_i^0-\nabla f_i(x^\star)\|^2, \qquad V_0=\|x^0-x^\star\|^2+\frac{9Nh^2}{S}C_0,C0​=N1​i∑​∥ci0​−∇fi​(x⋆)∥2,V0​=∥x0−x⋆∥2+S9Nh2​C0​, q=1−μh2,wr=q−(r+1),WT=∑r=0T−1wr.q=1-\frac{\mu h}{2},\qquad w_r=q^{-(r+1)},\qquad W_T=\sum_{r=0}^{T-1}w_r.q=1−2μh​,wr​=q−(r+1),WT​=r=0∑T−1​wr​.

For h≤1/(81β)h\le1/(81\beta)h≤1/(81β) and μh≤S/(15N)\mu h\le S/(15N)μh≤S/(15N), the target is

1WT∑r=0T−1wr E[f(xr)−f(x⋆)]≤V0hWT+12hσ2KS(1+Sηg2).\frac1{W_T}\sum_{r=0}^{T-1}w_r\, \mathbb E\bigl[f(x^r)-f(x^\star)\bigr] \le\frac{V_0}{hW_T} +\frac{12h\sigma^2}{KS}\left(1+\frac S{\eta_g^2}\right).WT​1​r=0∑T−1​wr​E[f(xr)−f(x⋆)]≤hWT​V0​​+KS12hσ2​(1+ηg2​S​).

This is a finite-round formulation of Appendix E.1, Lemma 15, PDF p. 30; its μ=0\mu=0μ=0 case is the first bound on PDF p. 31.

NonconvexFiniteRoundConvergence assumes a lower bound flowerf_\mathrm{lower}flower​ for fff and initializes every client control with KKK fresh gradients at x0x^0x0. Put F0=f(x0)−flowerF_0=f(x^0)-f_\mathrm{lower}F0​=f(x0)−flower​. For h≤(S/N)2/3/(24β)h\le(S/N)^{2/3}/(24\beta)h≤(S/N)2/3/(24β), establish

1T∑r=0T−1E∥∇f(xr)∥2≤14F0hT+70βhσ2KS(1+Sηg2).\frac1T\sum_{r=0}^{T-1}\mathbb E\|\nabla f(x^r)\|^2 \le\frac{14F_0}{hT} +\frac{70\beta h\sigma^2}{KS}\left(1+\frac S{\eta_g^2}\right).T1​r=0∑T−1​E∥∇f(xr)∥2≤hT14F0​​+KS70βhσ2​(1+ηg2​S​).

This is the explicit finite-round consequence targeted from Appendix E.2, Lemma 19, PDF p. 34, with the initialization specified on PDF pp. 31 and 35. The full-client warm start's communication cost is separate from these TTT optimization rounds.

What the result establishes

The bounds quantify optimization progress under stochastic gradients, several local steps, and partial participation. They permit arbitrarily different client objectives within the stated smoothness and convexity assumptions. Their output is a weighted random server iterate, represented by its expected loss, or a uniform random server iterate for the squared-gradient guarantee; this follows the output convention in equation (22), PDF p. 26.

The mathematical convergence analysis is published. The open work here is a Lean proof under the explicit stochastic-run model. The proposed statements do not themselves supply a machine-checked convergence proof. A completed development would provide reusable results about sampled finite averages, adaptive gradient queries, and optimization with stored noisy controls.

Where the difficulty lies

Local gradients are evaluated at different client states, and inactive clients retain controls computed in earlier rounds. Consequently, treating the server update as a centralized stochastic-gradient step discards both local disagreement and stale-control error. The challenge is to control these quantities while preserving their dependence on earlier randomness, as reflected in Appendix E.1–E.2, PDF pp. 26–35.

The source also needs careful transcription. The printed Theorem VII nonconvex noise term on PDF p. 25 differs from Lemma 19 in its smoothness and sampling factors, while output indices differ between equation (22) and the finite sum on p. 31. The exact targets above use the proof's finite-round constants and explicit indexing; they do not claim a verbatim formalization of every printed asymptotic rate.

Formalization scope

SCAFFOLD.Space is finite-dimensional real Euclidean space; SCAFFOLD.Problem records differentiable client objectives and smoothness. SCAFFOLD.Run records a standard Borel probability space, full-history filtration, measurable square-integrable iterates and oracle samples, and the concrete updates. Virtual local paths are generated before a fresh uniform size-SSS subset is sampled, as permitted by the paragraph after equation (22), PDF p. 26.

The conventions include S=NS=NS=N, K=1K=1K=1, zero noise, and μ=0\mu=0μ=0. The output indices are precisely 0,…,T−10,\ldots,T-10,…,T−1. Convex initial controls are deterministic; the nonconvex branch uses the specified stochastic warm start. The run definition assumes no descent inequality or convergence conclusion. Contributions should establish the four named propositions and necessary analysis/probability infrastructure while retaining these semantics.

Selected references

  • Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank J. Reddi, Sebastian U. Stich, and Ananda Theertha Suresh, SCAFFOLD: Stochastic Controlled Averaging for Federated Learning, ICML 2020, PMLR 119:5132–5143; arXiv:1910.06378v4, revised 2021. Primary scope: Theorem III, PDF p. 5; A3–A5 and (9), p. 14; Lemma 5, p. 17; Appendix E, pp. 25–35, especially Lemmas 15 and 19.
6 thms1 active userReviewed
🏆Completed
Convex OptimizationMachine Learning·Captain: Minghui

A Field Guide to Federated Optimization: Convex FedAvg ConvergenceResearch Paper

Why local training needs a convergence guarantee

Federated optimization studies learning when data and computation are spread across clients. Communicating after every stochastic gradient step can be costly, so clients often take several steps before averaging their models. The difficulty is that different clients can optimize different objective functions. Their models then move apart between communication rounds. A convergence guarantee must account for both stochastic gradient noise and this disagreement. Wang et al. give an explicit analysis of this tradeoff in A Field Guide to Federated Optimization, Section 6.1.

The relevant historical sequence is the introduction of FedAvg by McMahan et al. in 2017, the development of local-SGD convergence analyses reviewed by Wang et al., and the unified illustrative analysis in the 2021 field guide. The present target is that guide's known convex convergence theorem, rather than a new conjecture about arbitrary federated learning. The remaining open task is a Lean proof of the stated result. The mission is categorized as ResearchPaper: it formalizes a specific known result rather than proposing a new mathematical conjecture.

Setting: full participation with uniform weights

There are M≥1M\ge1M≥1 clients and a parameter vector in Rd\mathbb R^dRd. Client iii has a differentiable convex objective FiF_iFi​. The global objective is F(x)=M−1∑i=1MFi(x)F(x)=M^{-1}\sum_{i=1}^M F_i(x)F(x)=M−1∑i=1M​Fi​(x). Every local gradient is LLL-Lipschitz for the same L>0L>0L>0. Fix a global minimizer x⋆x^\starx⋆ of FFF and a deterministic initial model x0x_0x0​, and write D=∥x0−x⋆∥D=\|x_0-x^\star\|D=∥x0​−x⋆∥.

Every client participates in every round. Each of T≥1T\ge1T≥1 rounds consists of τ≥1\tau\ge1τ≥1 local steps with constant learning rate η\etaη. If xit,kx_i^{t,k}xit,k​ is client iii's state after kkk local steps of round ttt, its next state is xit,k+1=xit,k−ηgit,kx_i^{t,k+1}=x_i^{t,k}-\eta g_i^{t,k}xit,k+1​=xit,k​−ηgit,k​. At the next round all clients restart from the average of the preceding round's terminal states. The shadow iterate is xˉt,k=M−1∑ixit,k\bar x^{t,k}=M^{-1}\sum_i x_i^{t,k}xˉt,k=M−1∑i​xit,k​; it is defined even at local steps where clients do not communicate.

On a probability space (Ω,A,P)(\Omega,\mathcal A,\mathbb P)(Ω,A,P), the history before a step contains all past oracle draws. The stochastic gradients are conditionally unbiased, their conditional squared errors have expectation at most σ2\sigma^2σ2, and the different clients' current gradients are conditionally independent. The heterogeneity bound is ∥∇Fi(x)−∇F(x)∥≤ζ\|\nabla F_i(x)-\nabla F(x)\|\le\zeta∥∇Fi​(x)−∇F(x)∥≤ζ for every client and every point, where σ,ζ≥0\sigma,\zeta\ge0σ,ζ≥0. These are the assumptions of Section 6.1.1, PDF p. 40, equations (11)–(14), with the history and independence convention used explicitly in Appendix D.1 immediately after equation (27), PDF p. 87.

Formalization targets

The principal goal is Theorem 1, equation (15). For 0<η≤1/(4L)0<\eta\le1/(4L)0<η≤1/(4L), establish

E ⁣[1τT∑t=0T−1∑k=1τ(F(xˉt,k)−F(x⋆))]≤D22ητT+ησ2M+4τη2Lσ2+18τ2η2Lζ2.\mathbb E\!\left[\frac1{\tau T}\sum_{t=0}^{T-1}\sum_{k=1}^{\tau} \bigl(F(\bar x^{t,k})-F(x^\star)\bigr)\right] \le \frac{D^2}{2\eta\tau T}+\frac{\eta\sigma^2}{M} +4\tau\eta^2L\sigma^2+18\tau^2\eta^2L\zeta^2.E[τT1​t=0∑T−1​k=1∑τ​(F(xˉt,k)−F(x⋆))]≤2ητTD2​+Mησ2​+4τη2Lσ2+18τ2η2Lζ2.

Two source milestones describe the intermediate results. Lemma 1 bounds the conditional average loss within a round by the decrease of squared distance to the minimizer, plus a noise term and a sum of client disagreements. Lemma 2 bounds each conditional squared disagreement by 18τ2η2ζ2+4τη2σ218\tau^2\eta^2\zeta^2+4\tau\eta^2\sigma^218τ2η2ζ2+4τη2σ2. Both are stated on PDF p. 41, Section 6.1.2, with proofs in Appendix D, PDF pp. 86–88. They remain genuine open proof obligations; the algorithm model does not assume either estimate.

An additional milestone records the same theorem's tuned-step consequence, equations (16)–(17), in the regime D,σ,ζ>0D,\sigma,\zeta>0D,σ,ζ>0. With

η=min⁡{14L,MDτTσ,D2/3τ2/3T1/3L1/3σ2/3,D2/3τT1/3L1/3ζ2/3},\eta=\min\left\{\frac1{4L}, \frac{\sqrt M D}{\sqrt\tau\sqrt T\sigma}, \frac{D^{2/3}}{\tau^{2/3}T^{1/3}L^{1/3}\sigma^{2/3}}, \frac{D^{2/3}}{\tau T^{1/3}L^{1/3}\zeta^{2/3}}\right\},η=min{4L1​,τ​T​σM​D​,τ2/3T1/3L1/3σ2/3D2/3​,τT1/3L1/3ζ2/3D2/3​},

the same expected loss is at most

2LD2τT+2σDMτT+5L1/3σ2/3D4/3τ1/3T2/3+19L1/3ζ2/3D4/3T2/3.\frac{2LD^2}{\tau T}+\frac{2\sigma D}{\sqrt{M\tau T}} +\frac{5L^{1/3}\sigma^{2/3}D^{4/3}}{\tau^{1/3}T^{2/3}} +\frac{19L^{1/3}\zeta^{2/3}D^{4/3}}{T^{2/3}}.τT2LD2​+MτT​2σD​+τ1/3T2/35L1/3σ2/3D4/3​+T2/319L1/3ζ2/3D4/3​.

The principal goal includes zero-noise and zero-heterogeneity cases. The extra positivity conditions apply only to the printed tuned-step formula, whose denominators otherwise require separate conventions.

What this establishes

The bound separates an initial-distance term, a noise term improved by the number of clients, and two costs of local updates. It quantifies how local work interacts with stochastic noise and differing client objectives. Its conclusion concerns the average objective gap along the post-update shadow sequence; it does not assert the same bound for every last iterate or for the average of client losses. These distinctions follow directly from the quantity defined in equation (14).

A completed formalization would provide reusable checked components for stochastic optimization: finite client averages, history-conditioned oracle assumptions, per-round potential estimates, and disagreement bounds. The paper supplies the mathematical proof; this draft supplies checked statements and definitions. No convergence proof is claimed by creating or compiling the proposal.

Where the difficulty lies

The average update evaluates each gradient at its own client's state. It therefore does not directly equal a centralized stochastic-gradient step at the shadow iterate. A proof must control that discrepancy quantitatively, preserve the conditioning on the round's starting history, and justify the 1/M1/M1/M noise improvement using independent client sampling. Ignoring the sampling relationship can invalidate the advertised bound even for scalar quadratic objectives.

Formalization scope

The model uses finite-dimensional real Euclidean space, including the harmless zero-dimensional case, and a standard Borel probability space. A filtration indexed by tτ+kt\tau+ktτ+k records the full past. Local states and gradients carry explicit measurability and finite-second-moment conditions. These probability conventions support actual Bochner and conditional expectations; integrals are not treated as arbitrary total functions without analytic obligations.

The local objectives, global objective, gradients, iterates, and shadow averages are concrete functions. The model assumes neither a drift bound nor a progress bound. Source hypotheses are uniform in the model point, and the minimizer is an actual minimizer of the averaged objective. Unequal weighting, partial client participation, nonconvex objectives, adaptive step sizes, and privacy mechanisms are outside this particular theorem. Contributions should prove the named source lemmas or their necessary analytic infrastructure while retaining these statements.

Selected references

  • Jianyu Wang et al., A Field Guide to Federated Optimization, 2021, arXiv:2107.06917v1, Section 6.1.1–6.1.2, PDF pp. 40–41; Appendix D, PDF pp. 86–88.
  • H. Brendan McMahan et al., Communication-Efficient Learning of Deep Networks from Decentralized Data, AISTATS 2017, arXiv:1602.05629, the FedAvg algorithm cited by the field guide. This is historical context, not an additional target.
5 thms1 active userReviewed
Captain: mikedeng1

Markov Processes: Characterization and Convergence V: Change of Variables for Continuous SemimartingalesTextbook

Change of variables along random paths

The ordinary chain rule describes how a smooth function changes along a differentiable path. A continuous stochastic path can have fluctuations whose accumulated squared increments remain visible even when the time partition becomes arbitrarily fine. The change of variables formula must account for these fluctuations as well as the part of the path with finite variation. Ethier and Kurtz give this identity for time-dependent functions of multidimensional continuous semimartingales in Chapter 5, Theorem 2.9, equations (2.40)–(2.41), of Markov Processes: Characterization and Convergence.

Processes and smooth functions

Work on a complete probability space with a filtration, an increasing family of sigma algebras describing the information available at each nonnegative time. The initial sigma algebra contains every ambient null set. For each coordinate, a continuous semimartingale has a decomposition

Xi(t)=Xi(0)+Vi(t)+Mi(t).X_i(t)=X_i(0)+V_i(t)+M_i(t).Xi​(t)=Xi​(0)+Vi​(t)+Mi​(t).

The initial vector is measurable with respect to the initial information. Each process ViV_iVi​ is continuous and adapted, starts at zero, and has bounded variation on every bounded time interval. Each MiM_iMi​ is a continuous adapted local martingale, initially zero almost surely: stopping along a suitable sequence of stopping times tending to infinity gives martingales. No Brownian driver or diffusion coefficient is specified.

Let f(t,x)f(t,x)f(t,x) be a real-valued function of nonnegative time and a vector in Rd\mathbb R^dRd. The regularity class C1,2C^{1,2}C1,2 requires joint continuity of the function, its first time derivative, all first spatial derivatives, and all second spatial derivatives. The time derivative at zero is interpreted from the right. The hypotheses impose no compact support, global derivative bound, or second time derivative.

The change of variables identity

The goal is the complete identity

f(t,X(t))−f(0,X(0))=∫0tft(s,X(s)) ds+∑i∫0tfxi(s,X(s)) dVi(s)+∑i∫0tfxi(s,X(s)) dMi(s)+12∑i,j∫0tfxixj(s,X(s)) d⟨Mi,Mj⟩s.\begin{aligned} f(t,X(t))-f(0,X(0))={}&\int_0^t f_t(s,X(s))\,ds\\ &+\sum_i\int_0^t f_{x_i}(s,X(s))\,dV_i(s)\\ &+\sum_i\int_0^t f_{x_i}(s,X(s))\,dM_i(s)\\ &+\frac12\sum_{i,j}\int_0^t f_{x_ix_j}(s,X(s))\,d\langle M_i,M_j\rangle_s. \end{aligned}f(t,X(t))−f(0,X(0))=​∫0t​ft​(s,X(s))ds+i∑​∫0t​fxi​​(s,X(s))dVi​(s)+i∑​∫0t​fxi​​(s,X(s))dMi​(s)+21​i,j∑​∫0t​fxi​xj​​(s,X(s))d⟨Mi​,Mj​⟩s​.​

It holds almost surely simultaneously for every nonnegative time. The quadratic covariation ⟨Mi,Mj⟩\langle M_i,M_j\rangle⟨Mi​,Mj​⟩ records the limit of sums of products of increments. Both diagonal and off-diagonal contributions occur. The bracket processes and integral versions are supplied by the conclusion, rather than imposed as additional restrictions on the input processes.

What the identity supplies

The formula determines how a smooth observation of the evolving state changes. Its terms separate explicit time dependence, finite-variation motion, martingale fluctuations, and their second-order correction. This is the calculus result for a given continuous semimartingale; existence or uniqueness of solutions to a stochastic differential equation is a separate mathematical question. The target is the known theorem in the cited book. Its mathematical assertion includes existence of the integral versions appearing in the displayed identity.

Why continuity does not give the ordinary chain rule

Continuity of the sample paths does not imply bounded variation of the martingale component. Consequently, ordinary pathwise integration against that component does not supply the needed calculus. The stochastic integral requires a probabilistic limiting interpretation. A further issue is the exceptional set: an identity established separately at each time does not by itself provide one event of full probability on which it holds at every time. The continuous versions and the order of the quantifiers address that distinction.

Representations and boundary conventions

Coordinates are indexed by Fin d, and time is nonnegative. Allowing zero coordinates also includes the consistent case of a function of time alone. A dyadic partition has 2n2^n2n intervals, so the denominators in the defining sums are nonzero, including at partition level zero. At time zero all increments vanish.

The finite-variation integrals are characterized pathwise by limits of left dyadic sums. The stochastic integrals are continuous adapted processes characterized by convergence of those sums in probability at each fixed time. The cross brackets are characterized by the corresponding increment-product sums and are required to have continuous adapted versions of locally bounded variation. These predicates do not assume the change of variables identity. The ordinary time integral has a continuous integrand on each compact interval, so its integrability follows from the stated hypotheses. The source's almost-sure version conventions are retained; right continuity of the filtration is not an additional hypothesis.

Selected reference

Stewart N. Ethier and Thomas G. Kurtz, Markov Processes: Characterization and Convergence, Wiley, 1986. Chapter 5, Stochastic Integral Equations, §2, Theorem 2.9, printed p.287 (held PDF p.296). Probability-space and version conventions: printed pp.279–280 and 286 (PDF pp.288–289 and 295).

6 thms1 active userReviewed
Stochastic Systems·Captain: mikedeng1

Markov Processes: Characterization and Convergence IV: Martingale characterization of Markov processesTextbook

From local evolution to a Markov process

A stochastic process can be described through the way functions of its state change over time. For a suitable pair of functions f and g, the difference between f evaluated along the process and the accumulated integral of g has zero conditional drift. A martingale problem specifies a collection of such pairs. The question is whether these local conditions determine the evolution of the process, including the dependence of its future on its past. Ethier and Kurtz's Chapter 4, Theorem 4.1 answers this question under analytic hypotheses on the collection of pairs and a separation condition on the functions being observed.

State space, functions, and histories

Let E be a separable metric space, with its Borel sigma algebra. Write B(E) for the bounded real Borel functions, equipped with the uniform norm. A linear relation A is a linear subspace of B(E) × B(E). A pair (f,g) in A prescribes g as the drift associated with f; the relation is allowed to be multivalued. It is dissipative when

a∥f∥≤∥af−g∥((f,g)∈A, a>0).a\|f\|\leq\|af-g\|\qquad ((f,g)\in A,\ a>0).a∥f∥≤∥af−g∥((f,g)∈A, a>0).

Let μ be a probability measure on E. A solution X of the martingale problem for (A,μ) is a jointly measurable process on a probability space with initial law μ, satisfying the compensated moment identities of Chapter 4, equation (3.4). Those identities test each compensated increment against products of bounded Borel functions of finitely many earlier states. The natural past at time s is the sigma algebra generated by all coordinates X(r) with r≤s.

A subspace L of B(E) is separating if two Borel probability measures that have the same integral against every function in L must be equal. This is a condition on measures, stronger in purpose than merely distinguishing individual states.

Characterization target

Suppose A′ is a linear subrelation of A and, for some λ>0,

R(λ−A′)‾=D(A′)‾=L,\overline{\mathcal R(\lambda-A')}=\overline{\mathcal D(A')}=L,R(λ−A′)​=D(A′)​=L,

where both closures use the uniform norm and L is separating. Given a solution X of (A,μ), the target is the existence of a strongly continuous contraction semigroup T on L generated by the closed relation A′‾\overline{A'}A′, together with

E[f(X(s+t))∣FsX]=(T(t)f)(X(s))(s,t≥0, f∈L).\mathbb E[f(X(s+t))\mid\mathcal F_s^X]=(T(t)f)(X(s))\quad(s,t\geq0,\ f\in L).E[f(X(s+t))∣FsX​]=(T(t)f)(X(s))(s,t≥0, f∈L).

The process is Markov: for every bounded Borel observable, conditioning a future observation on the entire past agrees almost surely with conditioning on the present state. In addition, any other solution with initial law μ has exactly the same finite-dimensional distributions as X. The competing solution may live on a different probability space. Existence of X is an assumption; the existence conclusion concerns its corresponding semigroup.

What the characterization establishes

The theorem joins an analytic description of evolution with a probabilistic one. The closed generator relation determines a semigroup, while the martingale identities identify the conditional evolution of the supplied process. Separation allows observables in L to determine probability laws. Uniqueness concerns every finite list of observation times, which is the source's convention for this martingale problem.

This is a known theorem of Ethier and Kurtz. The formalization target is its complete statement and eventually a machine-checked proof. The present theorem remains a proof obligation. Its expression infrastructure consists of bounded functions, the natural past, and the finite-history definition of a solution.

Mathematical difficulty

The observed functions initially lie in a subspace L, whereas the Markov conclusion ranges over all bounded Borel observations. Identifying only individual expectations does not by itself establish conditional evolution or equality of joint laws. The analytic part must also identify the entire generator graph: agreement on the original relation alone is weaker than the asserted equality with its closure. Both closure hypotheses and probability-measure separation are therefore part of the target.

Formalization scope

Bounded functions are actual pointwise real functions with the uniform norm. Borel measurability is imposed on the relation, and uniform limits preserve it. The construction does not use functions modulo almost-everywhere equality. The state space is separable and metric; completeness and local compactness are not additional hypotheses. Processes need no right-continuous or cadlag paths.

Histories may be empty, repeated, or unordered, with every history time at most the start of the increment. Products combine repeated observations. Boundedness and finite time intervals ensure the required integrability. Nonnegative real times index the processes. The semigroup is represented by a real-indexed family whose axioms concern nonnegative times; negative-time values have no mathematical role. Generator equality is a biconditional for the derivative limit from positive times. The target retains semigroup correspondence, the ordinary Markov property, and unrestricted finite-dimensional uniqueness as one theorem.

Selected references

Stewart N. Ethier and Thomas G. Kurtz, Markov Processes: Characterization and Convergence, Wiley, 1986, Chapter 4, Theorem 4.1, printed p.182 (PDF p.191); equations (3.4) and (4.2), printed pp.174 and 183. Separation conventions: Chapter 3, printed pp.112 and 116. Chapter 4.

4 thms1 active userReviewed
Stochastic Systems·Captain: mikedeng1

Ethier–Kurtz: Martingales with partially ordered time (Theorem 8.7)Textbook

Martingales with partially ordered time

Optional sampling relates a process observed at two random times. A martingale has a conditional expectation at an earlier index equal to its value there, but this defining property initially concerns deterministic indices. Random indices require a separate theorem. In Chapter 2, Section 8, Ethier and Kurtz extend optional sampling to a partially ordered family of indices for use in the time changes of Chapter 6. The target is their Theorem 8.7, printed pages 87–88.

Metric lattices and information

A metric lattice is a partially ordered metric space I in which each pair of points has a greatest lower bound and a least upper bound. These are written as the meet and join. Both operations are jointly continuous. Points need not be comparable. The interval between u ≤ v consists of all points w with u ≤ w ≤ v.

A subset is separable from above if it contains a sequence aₙ that approximates each of its points w by the finite meets of those aᵢ with w ≤ aᵢ and i ≤ n. Each such meet is taken once this finite set is nonempty. The resulting sequence of meets must converge to w. The theorem assumes this condition for each interval.

Let Ω carry a probability measure P and an ambient sigma algebra. A filtration assigns a sub-sigma algebra Fᵤ to each index u, increasing with the order. A real-valued process X is a martingale if each X(u) is Fᵤ-measurable and integrable and, whenever u ≤ v,

E[X(v)∣Fu]=X(u)almost surely.E[X(v)\mid F_u]=X(u)\quad\text{almost surely}.E[X(v)∣Fu​]=X(u)almost surely.

Right continuity here has a specific lattice meaning: for every u and every outcome ω, X(u ∨ v,ω) tends to X(u,ω) as v tends to u. A stopping time τ is a Borel-measurable I-valued random variable such that {τ ≤ u} belongs to Fᵤ for every u.

The optional-sampling target

Take stopping times τ₁ ≤ τ₂ pointwise. Suppose there are deterministic sequences uₙ and vₙ in I such that

P{un≤τ1≤τ2≤vn}⟶1,P\{u_n\leq\tau_1\leq\tau_2\leq v_n\}\longrightarrow1,P{un​≤τ1​≤τ2​≤vn​}⟶1,

and

E[∣X(vn)∣1{τ2≤vn}c]⟶0.E[|X(v_n)|\mathbf1_{\{\tau_2\leq v_n\}^{c}}]\longrightarrow0.E[∣X(vn​)∣1{τ2​≤vn​}c​]⟶0.

Assume also that X(τ₂) is integrable. The target is

E[X(τ2)∣Fτ1]=X(τ1)almost surely.E[X(\tau_2)\mid F_{\tau_1}]=X(\tau_1)\quad\text{almost surely}.E[X(τ2​)∣Fτ1​​]=X(τ1​)almost surely.

The sigma algebra Fτ consists of ambient-measurable events A for which A ∩ {τ ≤ u} belongs to Fᵤ for every u. This is the information used in the conclusion.

What the result provides

The conclusion extends the martingale identity to random lattice indices under explicit exhaustion and tail assumptions. Neither a deterministic bound on both stopping times nor a linear ordering of all indices is part of the statement. This is a known theorem in the cited book. The formal target retains the full conditional-expectation identity; its proof remains to be formalized.

Why the hypotheses matter

In a partially ordered space, the complement of {τ₂ ≤ vₙ} includes incomparable indices. Replacing this complement by {vₙ < τ₂} would change the tail assumption. Likewise, right continuity must use lattice joins, rather than a one-dimensional time convention. Controlling probabilities of the exhaustion events alone does not state the integrability control required by the second limit. Both limits belong to the theorem.

Mathematical scope and conventions

The index space has a metric and a lattice structure with continuous meet and join. No top, bottom, completeness of the metric, or linear order is required. The filtration need not be complete or right continuous. The stopping times are finite I-valued variables; their measurability is explicit. The sequences uₙ and vₙ need not be monotone. The nonnegative tail expectation uses extended nonnegative integration, avoiding a default value for a nonintegrable real integral. Indexwise integrability is retained explicitly, along with terminal integrability.

The two accompanying definitions express separation from above and the stopped sigma algebra. Together with the martingale and stopping-time conditions, they specify the single optional-sampling theorem. The target contains no additional supporting-theorem milestones.

Source

Stewart N. Ethier and Thomas G. Kurtz, Markov Processes: Characterization and Convergence, Wiley, 1986, Chapter 2 §8, Theorem 8.7, printed pp. 87–88 (supplied PDF pp. 96–97); definitions and equation (8.6), printed p. 85 (PDF p. 94). Chapter 2.

3 thms1 active userReviewed
Captain: mikedeng1

Markov Processes: Characterization and Convergence 20: Two-type critical branching limitsTextbook

Why two types change the limit

Critical branching is a basic scaling model for populations whose expected size neither grows nor decays at leading order. A single-type population has one macroscopic direction, but a two-type population has two coupled directions. In the critical regime considered by Ethier and Kurtz, one weighted population mode survives on the diffusive scale while a second mode is pulled rapidly toward zero. The rapid mode still leaves a random integrated contribution and an initial boundary layer, so discarding it would lose part of the limiting dynamics. The mission packages all of these conclusions from Chapter 9, Section 2, Theorem 2.1 of Ethier and Kurtz as one formal target.

The branching model

The state is a pair of natural numbers recording the numbers of particles of types 1 and 2. A type-iii particle lives for an exponential time with positive rate λi\lambda_iλi​. At death it is replaced by a random pair of offspring with law ρi\rho_iρi​. The generator therefore applies the type-specific replacement rule at an intensity proportional to the current number of particles of that type. The formal predicate IsTwoTypeBranching records a measurable càdlàg process satisfying the corresponding natural-past martingale identities on finite-support tests, with an explicit integrability condition.

The offspring mean matrix has strictly positive entries. Its critical weighted rate matrix has a positive eigenvector ν\nuν with eigenvalue zero and an opposite-sign eigenvector μ\muμ with eigenvalue −η-\eta−η, where η>0\eta>0η>0. For the nnnth process, Lean consistently uses the positive index n+1n+1n+1. At accelerated time (n+1)t(n+1)t(n+1)t, the scaled modes are

Xn(t)=ν1Z1(n)((n+1)t)+ν2Z2(n)((n+1)t)n+1,Yn(t)=μ1Z1(n)((n+1)t)+μ2Z2(n)((n+1)t)n+1.X_n(t)=\frac{\nu_1 Z^{(n)}_1((n+1)t)+\nu_2 Z^{(n)}_2((n+1)t)}{n+1}, \qquad Y_n(t)=\frac{\mu_1 Z^{(n)}_1((n+1)t)+\mu_2 Z^{(n)}_2((n+1)t)}{n+1}.Xn​(t)=n+1ν1​Z1(n)​((n+1)t)+ν2​Z2(n)​((n+1)t)​,Yn​(t)=n+1μ1​Z1(n)​((n+1)t)+μ2​Z2(n)​((n+1)t)​.

The compensated fast coordinate is

Wn(t)=Yn(t)+∫0t(n+1)ηYn(s) ds.W_n(t)=Y_n(t)+\int_0^t (n+1)\eta Y_n(s)\,ds.Wn​(t)=Yn​(t)+∫0t​(n+1)ηYn​(s)ds.

Formalization target

The goal is the complete three-part statement of Theorem 2.1. First, (Xn,Wn)(X_n,W_n)(Xn​,Wn​) converges jointly in path law to a continuous two-dimensional diffusion (X,W)(X,W)(X,W). Its covariance matrix is built from the raw second moments of the two offspring replacement increments, and its generator has the form

Af(x,w)=x2(a11fxx+2a12fxw+a22fww)(x,w).Af(x,w)=\frac{x}{2}\left(a_{11}f_{xx}+2a_{12}f_{xw}+a_{22}f_{ww}\right)(x,w).Af(x,w)=2x​(a11​fxx​+2a12​fxw​+a22​fww​)(x,w).

Second, on every finite time interval the fast mode is uniformly close in probability to its exponentially decaying initial layer Yn(0)e−(n+1)ηtY_n(0)e^{-(n+1)\eta t}Yn​(0)e−(n+1)ηt. On every interval 0<t1<t20<t_1<t_20<t1​<t2​, its integrated drift converges weakly to W(t2)−W(t1)W(t_2)-W(t_1)W(t2​)−W(t1​). Third, the first population coordinate is uniformly reconstructed from XnX_nXn​ together with that same initial layer. The limit statement keeps the possibly nonzero initial fast coordinate and the separate nonnegativity conclusion for the slow coordinate.

What the result supplies

The theorem identifies both the persistent and collapsing directions of a critical multitype system. The slow direction produces population-scale diffusion, while the compensated fast direction records fluctuations that remain visible after the raw fast mode contracts. The reconstruction formula connects these mode coordinates back to the original particle count. Keeping the path limit, fast decay, integrated limit, and reconstruction together prevents an apparently simpler marginal statement from omitting the initial-layer behavior needed at time zero.

The textbook theorem is already mathematically proved. This mission asks for a Lean proof of the reviewed formal statement; the uploaded goal is intentionally a statement with a proof placeholder, not a claim of machine-checked completion. The concrete generator, branching-law predicate, scaling map, càdlàg condition, and diffusion martingale-problem predicate provide reusable vocabulary for related multitype limit theorems.

Where the formal difficulty lies

Coordinatewise convergence is insufficient. The result couples full path laws, a continuous diffusion martingale problem, compact-uniform control of an initial layer, and a positive-time integral of a rapidly changing mode. The naive step of simply setting the fast mode to zero fails at time zero and erases the exponential term needed in the population reconstruction. The covariance coefficients also come from replacement increments rather than centered offspring counts, so changing that convention alters the limiting operator.

Formalization scope and conventions

The state space is exactly N×N\mathbb N\times\mathbb NN×N; both offspring laws are probability measures, both coordinates have finite third moments under each parent law, and all four entries of the mean matrix are positive. Initial populations are deterministic floors of (n+1)zi(n+1)z_i(n+1)zi​ for nonnegative densities ziz_izi​. The positive indexing avoids division by zero in every scaling, clock, and exponential. Natural-number subtraction in the generator is used only with a zero event intensity when the relevant population is empty.

Path convergence is represented by a joint probability-space realization carrying the correct complete prelimit path laws and almost-sure uniform convergence on compact intervals to a continuous limit. This is the continuous-limit Skorohod convention used in the reviewed development. The definitions are concrete: none contains the target theorem or assumes its conclusions. The mission includes only expression-essential dependencies, reusing exact private book-wide definitions for càdlàg paths and the continuous diffusion generator law. It does not add proof-only lemmas or split the three source conclusions into separate missions.

Selected references

  • Stewart N. Ethier and Thomas G. Kurtz, Markov Processes: Characterization and Convergence, Wiley, 1986, Chapter 9, Section 2, Theorem 2.1, printed pp. 392–393. DOI
7 thms1 active userReviewed
Functional Analysis·Captain: mikedeng1

Markov Processes: Characterization and Convergence 18: Countable spin-flip systemsTextbook

Why infinite spin systems need a generation theorem

A spin-flip system models a countable collection of two-state components whose transition rates may depend on the entire current configuration. Such systems are basic examples of interacting particle processes: each local move is simple, but infinitely many possible moves may be active and their rates interact through the configuration. The central analytic question is whether the formal sum of local flip operators determines a genuine time evolution on continuous observables. Chapter 8, Section 3 of Stewart N. Ethier and Thomas G. Kurtz, Markov Processes: Characterization and Convergence, answers this question under uniform rate and influence bounds Ethier--Kurtz, Theorem 3.5.

This mission states that result without replacing the countable system by a finite-state chain or assuming finite total jump intensity. It also retains the theorem's cylinder-function core, which identifies finitely coordinate-dependent observables as a sufficient starting class for recovering the closed generator.

Configuration space and coordinate variation

Let S be a countable type of sites. A configuration is a function η : S → Bool; false and true relabel the source spins -1 and 1. The product topology on these configurations is the one induced by the discrete topology on each coordinate. The Lean space C(S → Bool, ℝ) consists of real-valued continuous observables on this compact product space and carries its uniform norm.

For a site i, spinFlip i η agrees with η away from i and negates the Boolean value at i. For an observable f, its coordinate variation at i is

var⁡i(f)=sup⁡η∣f(flip⁡iη)−f(η)∣.\operatorname{var}_i(f)=\sup_{\eta}\left|f(\operatorname{flip}_i\eta)-f(\eta)\right|.vari​(f)=ηsup​∣f(flipi​η)−f(η)∣.

The domain used in the theorem is the full class of continuous observables for which ∑ i, var_i(f) is summable. This is a one-coordinate variation domain: it measures the effect of changing one spin, rather than exchanging two occupied sites.

For every site i, the flip rate c i is itself a continuous function of the configuration. Rates are nonnegative and uniformly bounded in the uniform norm. Their dependence on other coordinates is controlled by a second uniform bound: for each i, the series ∑ j, var_j(c i) is summable, with a bound independent of i. These are the two clauses of equation (3.24).

Formalization target

The pre-generator graph implements equation (3.25). For every pair (f,g) in that graph, f has summable coordinate variation and

g(η)=∑ici(η)(f(flip⁡iη)−f(η)).g(\eta)=\sum_i c_i(\eta)\bigl(f(\operatorname{flip}_i\eta)-f(\eta)\bigr).g(η)=i∑​ci​(η)(f(flipi​η)−f(η)).

The goal is Theorem 3.5 in full. It asserts that every function in the prescribed domain has a continuous image in the graph; the closure of the graph in the product uniform-norm topology is single-valued; and that closure is exactly the generator graph of a strongly continuous contraction semigroup. The semigroup is positive and preserves the constant function one, giving the conservative Feller conclusion used by the source.

The final clause retains the source's cylinder core. Restrict the initial graph to functions depending on a finite set of coordinates, expressed by the existence of a finite set F such that agreement on F forces equal function values. The closure of this restricted graph equals the full generator graph. Generation and the core are kept together because both are conclusions of the same theorem.

Significance of the result

The result turns an infinite formal sum of local moves into a closed, single-valued Markov generator. This is stronger than merely defining the pointwise series: it supplies the strongly continuous positive contraction evolution, conservativity, and the derivative characterization of the entire generator domain. The cylinder-core clause means that finite-coordinate observables are graph-norm dense enough to recover that generator, even though the state space and the total collection of possible flips are countable.

The theorem is already proved in the source. The formalization target is its precise Lean statement, including the infinite-site domain, the uniform influence hypothesis, generator closure, and core. The reusable infrastructure consists of the book-wide strongly continuous contraction-semigroup predicate and the mission-local flip, variation, and graph definitions.

Where the analytic difficulty lies

The naive finite-rate jump argument does not apply. A uniform bound on each individual rate does not bound the sum of rates over a countably infinite site set, so the system may have infinite total potential flip intensity. Pointwise notation for the generator also does not by itself show that its image is continuous or that its graph closure is single-valued. The coordinate-influence bound and summable-variation domain are therefore essential parts of the target rather than optional regularity decoration.

The neighboring exclusion-process theorem is not interchangeable with this one. A spin flip changes one coordinate and permits creation or destruction of occupation, whereas exclusion dynamics exchanges two coordinates and conserves particle number. Their domains and rate conditions are correspondingly different.

Formalization scope

The Lean statement allows empty, finite, and countably infinite S; it does not impose nonemptiness or finiteness. Bool is an exact two-state relabeling, not a restriction to a special numerical spin convention. spinVariation uses a real supremum over the nonempty compact configuration space. Explicit Summable hypotheses prevent divergent real tsum expressions from receiving an unintended default interpretation.

The operator graph uses the actual pointwise series and requires a continuous output. Closure is taken in C(S → Bool, ℝ) × C(S → Bool, ℝ). The semigroup is indexed by real time, but every law and bound uses nonnegative times; negative-time values carry no mathematical requirement. No finite-total-rate hypothesis, finite-site approximation, two-site exchange dynamics, or weakened core statement is admitted.

Selected references

  • Stewart N. Ethier and Thomas G. Kurtz, Markov Processes: Characterization and Convergence, Wiley, 1986, Chapter 1, Section 3; Chapter 4; Chapter 8, Section 3, Theorem 3.5, equations (3.24)--(3.25), printed p. 381. Wiley DOI
5 thms1 active userReviewed
Functional Analysis·Captain: mikedeng1

Markov Processes: Characterization and Convergence 17: Nonlocal jump and Lévy generatorsTextbook

Why nonlocal generators matter

Markov processes with jumps model changes that cannot be represented by continuous diffusion paths: arrivals in queueing systems, sudden failures, population events, and discontinuous changes in financial or physical systems. Their infinitesimal descriptions are nonlocal generators, because the value of the operator at a state depends on test-function values at displaced states rather than only on derivatives at the current point. Chapter 8, Section 3 of Stewart N. Ethier and Thomas G. Kurtz, Markov Processes: Characterization and Convergence (Wiley, 1986), treats three complementary regimes. Theorem 3.1 covers finite-rate jumps on a general locally compact state space. Theorem 3.4 treats homogeneous Lévy operators on Euclidean space, including infinite jump activity and degenerate covariance. Theorem 3.3 gives the main target here: well-posedness for a time- and state-dependent compensated Lévy-type martingale problem.

These results connect concrete jump kernels and integro-differential operators to two standard descriptions of stochastic dynamics. Autonomous operators are realized as generators of conservative Feller semigroups, while time-dependent operators are characterized through martingale identities. Keeping all three source results in one mission records the nonlocal-generator theme without claiming that their distinct hypotheses imply one another.

The setting

For the Euclidean results, the state space is Rd\mathbb R^dRd, represented in Lean as EuclideanSpace ℝ (Fin d), with d>0d>0d>0. A covariance field a(t,x)a(t,x)a(t,x) is a continuous linear endomorphism, a drift field b(t,x)b(t,x)b(t,x) is a vector, and ν(t,x,dy)\nu(t,x,dy)ν(t,x,dy) is a measure of jump displacements. For a smooth compactly supported test function fff, the compensated Lévy-type operator is

Ltf(x)=12∑i,jaij(t,x) ∂ijf(x)+Df(x)[b(t,x)]+∫ ⁣(f(x+y)−f(x)−Df(x)[y]1+∥y∥2)ν(t,x,dy).L_t f(x)=\frac12\sum_{i,j}a_{ij}(t,x)\,\partial_{ij}f(x) +Df(x)[b(t,x)] +\int\!\left(f(x+y)-f(x)-\frac{Df(x)[y]}{1+\lVert y\rVert^2}\right)\nu(t,x,dy).Lt​f(x)=21​i,j∑​aij​(t,x)∂ij​f(x)+Df(x)[b(t,x)]+∫(f(x+y)−f(x)−1+∥y∥2Df(x)[y]​)ν(t,x,dy).

The denominator in the compensation term is part of the source convention. The covariance factor 1/21/21/2 applies only to the second-order term; the drift is unscaled. The weighted moment ∥y∥2/(1+∥y∥2)\lVert y\rVert^2/(1+\lVert y\rVert^2)∥y∥2/(1+∥y∥2) permits measures with infinite total mass while controlling the compensated integral.

A solution of the Lévy-type martingale problem is a jointly measurable process with the prescribed initial pushforward law whose integrated-generator increments have zero expectation against every bounded continuous function of any finite history. Smooth compactly supported functions are the spatial tests. This definition imposes no path-continuity requirement.

The finite-rate model uses a locally compact, noncompact, separable metric space EEE, a nonnegative rate λ(x)\lambda(x)λ(x), and a weakly continuous probability transition kernel μ(x,dy)\mu(x,dy)μ(x,dy). Its operator is Af(x)=λ(x)∫(f(y)−f(x))μ(x,dy)Af(x)=\lambda(x)\int(f(y)-f(x))\mu(x,dy)Af(x)=λ(x)∫(f(y)−f(x))μ(x,dy). Positive weights γ\gammaγ and η\etaη specify the graph domain and control behavior at infinity.

Formalization targets

Goal: nonautonomous Lévy-type well-posedness

The main target is Chapter 8, Section 3, Theorem 3.3. The covariance is continuous, symmetric, bounded, and strictly positive definite at every nonzero vector, without a uniform ellipticity constant. The drift is measurable and bounded. The weighted jump measure is integrable, and for every measurable set its weighted mass is a bounded continuous function of time and state.

For every initial probability measure, the conclusion asserts existence of a measurable-process solution on some probability space and uniqueness of all finite-dimensional distributions among every solution on every probability space. Setwise continuity is retained exactly; it is not replaced by weak convergence of measures or by finite total jump intensity.

Related target: homogeneous Lévy generation

Theorem 3.4 freezes the coefficients in time and space. The covariance may be degenerate but is symmetric and nonnegative, and the jump measure needs only the finite weighted integral. The full C^2\widehat C^2C2 graph consists of functions, first derivatives, and second derivatives that vanish at infinity. Its closure is the exact generator of a positive conservative strongly continuous contraction semigroup on C0(Rd)C_0(\mathbb R^d)C0​(Rd), and smooth compactly supported functions form a core.

Related target: finite-rate jump generation

Theorem 3.1 retains the general-state-space jump model. The transition kernel is both setwise measurable and weakly continuous; the rate and weights satisfy the source's vanishing and signed integral bounds. The weighted graph closes to a single-valued conservative Feller generator, and compactly supported continuous functions form a core.

Significance

The main theorem says that the displayed nonlocal characteristics determine stochastic dynamics in distribution even when the drift is only measurable and jump activity need not be finite. The two autonomous theorems identify concrete domains whose closures are genuine Feller generators, including positivity, contraction, strong continuity, conservativity, and core statements. Together they distinguish finite-rate kernels, homogeneous compensated Lévy dynamics, and nonautonomous Lévy-type dynamics while sharing the same generator language.

The formal contribution is a machine-checkable statement layer, not a proof claim. It exposes the exact compensation, weighted-integrability, graph-domain, semigroup, and finite-history conventions on which later proofs depend. The common diffusion operator, bounded-pointwise closure, and strongly continuous contraction-semigroup predicate are reused from earlier private missions in the same textbook series rather than duplicated under conflicting identifiers.

Where the difficulty lies

The obvious finite-jump approximation does not by itself preserve the full conclusions. Infinite jump activity requires compensation near zero, and convergence of the associated operators must still identify the intended martingale problem or the exact closed Feller generator. In the nonautonomous theorem, setwise continuity of weighted jump masses must coexist with merely measurable drift, while uniqueness ranges over arbitrary probability-space realizations and all finite-dimensional laws. Strengthening to continuous drift, imposing finite total jump intensity, assuming uniform ellipticity, or comparing only continuous-path solutions would materially weaken or alter the source theorem.

Formalization scope and conventions

Nonnegative real time is represented by ℝ≥0; Euclidean coordinates use Fin d. Fréchet derivatives express both gradient and Hessian terms. The jump measure is a measure of displacements yyy, so the operator evaluates f(x+y)f(x+y)f(x+y). Integrable hypotheses prevent Lean's totalized integral from silently assigning zero to a divergent positive weighted moment. Positive dimension is explicit through [NeZero d].

The martingale predicate includes empty finite histories and repeated observation times. Its bounded continuous history tests determine the same finite-dimensional identities used by the source, and no sample-path regularity is inserted. The autonomous graphs live in C0×C0C_0\times C_0C0​×C0​; bounded-pointwise closure means closure under uniformly bounded pointwise sequential limits, not ordinary norm closure. The finite-rate theorem preserves its restriction of the additional integrability clause to states with positive rate. No vacuous solution predicate, unweighted moment assumption, hidden nonzero-rate condition, or proof-only theorem dependency is introduced.

Selected references

  • Stewart N. Ethier and Thomas G. Kurtz, Markov Processes: Characterization and Convergence, Wiley, 1986, Chapter 8, Section 3, Theorems 3.1, 3.3, and 3.4. Wiley DOI
  • Stewart N. Ethier and Thomas G. Kurtz, same volume, Chapter 4, Sections 2–3 for Feller and martingale-problem conventions, and Appendix 3 for bounded-pointwise closure. Wiley DOI
12 thms1 active userReviewed
Functional Analysis·Captain: mikedeng1

Markov Processes: Characterization and Convergence 16: Degenerate diffusion on a simplexTextbook

Why simplex diffusions matter

Finite-dimensional simplices are the natural state spaces for proportions: allele frequencies, population shares, and other collections of nonnegative coordinates whose total cannot exceed one. A diffusion on such a space must do more than solve an unconstrained stochastic equation. Its covariance degenerates on the boundary, and its drift must respect every face so that the process remains in the simplex. Chapter 8 of Ethier and Kurtz develops generator results for this setting as part of the analytic foundation for Markov-process models with constrained state spaces Ethier and Kurtz, Chapter 8, §2.

This mission formalizes their Theorem 2.8, a known generation theorem rather than an open conjecture. The theorem treats arbitrary Lipschitz drift satisfying the inward boundary inequalities. It identifies the closed diffusion graph as a Feller generator and also states that polynomial restrictions form a core. The generality of the drift distinguishes this result from later population-model applications with particular mutation or selection coefficients.

The simplex and its boundary

For a positive integer ddd, the simplex state space is

Kd={x∈Rd:xi≥0 for every i,∑i=1dxi≤1}.K_d=\left\{x\in\mathbb R^d: x_i\ge 0\ \text{for every }i, \quad \sum_{i=1}^d x_i\le 1\right\}.Kd​={x∈Rd:xi​≥0 for every i,i=1∑d​xi​≤1}.

The missing mass 1−∑ixi1-\sum_i x_i1−∑i​xi​ may be viewed as an additional coordinate. In Lean, WFState d is the subtype of functions Fin d → ℝ satisfying precisely nonnegativity and the upper bound on the coordinate sum. These conditions already imply xi≤1x_i\le 1xi​≤1 for every coordinate.

Let b:Kd→Rdb:K_d\to\mathbb R^db:Kd​→Rd be the drift. The boundary conditions say that bi(x)≥0b_i(x)\ge 0bi​(x)≥0 whenever xi=0x_i=0xi​=0, while ∑ibi(x)≤0\sum_i b_i(x)\le 0∑i​bi​(x)≤0 whenever ∑ixi=1\sum_i x_i=1∑i​xi​=1. Thus the drift cannot point outward through a coordinate face or through the top face. No mutation–selection formula or extra smoothness condition is imposed: global Lipschitz continuity and these face inequalities are the complete drift hypotheses retained here.

The operator and target theorem

The covariance matrix is

aij(x)=xi(δij−xj),a_{ij}(x)=x_i(\delta_{ij}-x_j),aij​(x)=xi​(δij​−xj​),

and the associated simplex diffusion operator is

Gf(x)=12∑i,jxi(δij−xj) ∂i∂jf(x)+∑ibi(x) ∂if(x).Gf(x)=\frac12\sum_{i,j}x_i(\delta_{ij}-x_j) \,\partial_i\partial_j f(x)+\sum_i b_i(x)\,\partial_i f(x).Gf(x)=21​i,j∑​xi​(δij​−xj​)∂i​∂j​f(x)+i∑​bi​(x)∂i​f(x).

Only the second-order sum is multiplied by 1/21/21/2. The covariance becomes singular on boundary faces, so this is not an immediate instance of a uniformly elliptic whole-space theorem.

The formal target asserts that the uniform graph closure of

{(f,Gf):f∈C2(Kd)}\{(f,Gf):f\in C^2(K_d)\}{(f,Gf):f∈C2(Kd​)}

is single-valued and is exactly the generator graph of a positive, conservative, strongly continuous contraction semigroup on C(Kd)C(K_d)C(Kd​). Conservativity is expressed by preservation of the constant function 111. The generator is identified by the right derivative at time zero, not merely by containment of a convenient operator restriction.

The final clause says that polynomials restricted to KdK_dKd​ are a core: closing the graph obtained from polynomial first coordinates gives the same full generator graph. This is a graph-closure statement and is stronger than uniform density of polynomials as functions.

What the result provides

Single-valuedness shows that the closed graph behaves as an operator rather than a multivalued relation. Generation supplies a Feller semigroup whose positivity and preservation of constants give the Markov interpretation. The exact generator biconditional fixes the full infinitesimal domain, while the polynomial-core conclusion permits generator questions to be reduced to algebraically structured test functions without changing the closed operator.

The source theorem is already proved in the book. The formalization task is to recover its generator statement and core conclusion in Lean with all boundary, closure, and semigroup conventions explicit. The current mission contributes a source-reviewed statement and its expression-level definitions; it does not claim a completed Lean proof.

Why the formalization is delicate

The obvious route through standard elliptic diffusion generation does not apply because a(x)a(x)a(x) degenerates as coordinates approach the boundary. It is also insufficient to prove only that the displayed differential operator is contained in some generator: the theorem identifies the closure of the entire C2(Kd)C^2(K_d)C2(Kd​) graph and separately asserts the polynomial core.

Boundary semantics create another source of possible weakening. Dropping either inward condition would permit outward drift at a face. Replacing the arbitrary Lipschitz drift by a special population-genetics formula would narrow the theorem. Replacing graph equality by inclusion, or polynomial graph density by ordinary function density, would lose a stated conclusion. These distinctions are all preserved in the target.

Formalization scope

The Lean development uses the book-wide namespace EthierKurtz. The compact space C(Kd)C(K_d)C(Kd​) is represented by real-valued bounded continuous functions on WFState d; compactness makes this the intended continuous-function space with the uniform norm. A C2(Kd)C^2(K_d)C2(Kd​) function is represented through a globally twice continuously differentiable extension on the ambient finite-coordinate space, following the book’s Appendix 6 extension convention for closed convex sets.

The operator uses Fréchet derivatives evaluated on coordinate vectors, which represent the first and second partial derivatives in the finite-dimensional ambient space. The graph is closed in the product uniform norm. IsStronglyContinuousContractionSemigroup records identity at zero, the nonnegative-time semigroup law, contraction, and strong right continuity at zero. The target additionally records positivity, preservation of one, and equality between its derivative graph and the closed diffusion graph.

The scope excludes the zero-dimensional case, as required by 0 < d. It does not introduce a continuous-path hypothesis, a special drift parameterization, or uniform ellipticity. The only packaged declarations beyond the goal are the state space, operator, graph, and the already published semigroup predicate needed to state it. Proof-only lemmas and unrelated Chapter 8 results are outside this mission.

Selected references

  • Stewart N. Ethier and Thomas G. Kurtz, Markov Processes: Characterization and Convergence, Wiley, 1986, Chapter 8, §2, Theorem 2.8, printed p. 375; equation (1.15), printed p. 368; Appendix 6, printed pp. 499–500. DOI
5 thms1 active userReviewed
Functional Analysis·Captain: mikedeng1

Markov Processes: Characterization and Convergence 14: One-dimensional boundary classificationTextbook

Why endpoint classification matters

A one-dimensional diffusion is governed in the interior by a second-order differential operator, but the interior coefficients alone do not determine what happens when the process approaches the edge of its state space. An endpoint may be reachable from the interior or inaccessible; after arrival it may absorb, reflect, or retain the process according to a boundary parameter. The same issue persists when an endpoint is infinite. Chapter 8 of Stewart N. Ethier and Thomas G. Kurtz, Markov Processes: Characterization and Convergence (Wiley, 1986), organizes these alternatives through scale and speed integrals and identifies the exact operator domain that generates the corresponding Feller evolution. This mission formalizes Chapter 8, Section 1, Theorem 1.1, printed page 367.

The setting

Let −∞≤r0<r1≤∞-\infty\le r_0<r_1\le\infty−∞≤r0​<r1​≤∞. The real state interval is I=[r0,r1]∩RI=[r_0,r_1]\cap\mathbb RI=[r0​,r1​]∩R, its interior is I∘=(r0,r1)I^\circ=(r_0,r_1)I∘=(r0​,r1​), and Iˉ\bar IIˉ is the closed interval in the extended real line. A function in C(Iˉ)C(\bar I)C(Iˉ) is represented by a continuous function on that compact extended interval, so finite endpoint limits are part of the object even when an endpoint is infinite.

The interior diffusion operator is

Gf(x)=a(x)2f′′(x)+b(x)f′(x),Gf(x)=\frac{a(x)}2 f''(x)+b(x)f'(x),Gf(x)=2a(x)​f′′(x)+b(x)f′(x),

where aaa and bbb are continuous on I∘I^\circI∘ and a(x)>0a(x)>0a(x)>0 there. Choose an interior reference point rrr. Define

B(x)=∫rx2b(y)a(y) dy,p(x)=∫rxe−B(y) dy,m(x)=∫rx2eB(y)a(y) dy.B(x)=\int_r^x \frac{2b(y)}{a(y)}\,dy, \qquad p(x)=\int_r^x e^{-B(y)}\,dy, \qquad m(x)=\int_r^x \frac{2e^{B(y)}}{a(y)}\,dy.B(x)=∫rx​a(y)2b(y)​dy,p(x)=∫rx​e−B(y)dy,m(x)=∫rx​a(y)2eB(y)​dy.

The corresponding endpoint tests uuu and vvv are nonnegative extended-real integrals. Their values may be +∞+\infty+∞; preserving that possibility is essential because the four boundary classes are distinguished precisely by which of the two tests are finite.

Formalization target

Theorem 1.1 — one-dimensional boundary diffusion generation

At either endpoint, the pair (u,v)(u,v)(u,v) gives the Feller boundary class:

classuvregular<∞<∞exit<∞=∞entrance=∞<∞natural=∞=∞.\begin{array}{c|cc} \text{class} & u & v\\ \hline \text{regular} & <\infty & <\infty\\ \text{exit} & <\infty & =\infty\\ \text{entrance} & =\infty & <\infty\\ \text{natural} & =\infty & =\infty. \end{array}classregularexitentrancenatural​u<∞<∞=∞=∞​v<∞=∞<∞=∞.​​

Entrance and natural endpoints are inaccessible, so the generator domain imposes no boundary equation there. At an exit endpoint the limiting generator value must be zero. At a regular endpoint rir_iri​, a parameter qi∈[0,1]q_i\in[0,1]qi​∈[0,1] determines the boundary condition

qilim⁡x→riGf(x)=(−1)i(1−qi)lim⁡x→rieB(x)f′(x).q_i\lim_{x\to r_i}Gf(x) =(-1)^i(1-q_i)\lim_{x\to r_i}e^{B(x)}f'(x).qi​x→ri​lim​Gf(x)=(−1)i(1−qi​)x→ri​lim​eB(x)f′(x).

The target states that the graph consisting of f∈C(Iˉ)f\in C(\bar I)f∈C(Iˉ) that are twice continuously differentiable in the interior, whose GfGfGf extends continuously to Iˉ\bar IIˉ, and that satisfy the applicable condition at both endpoints is exactly the infinitesimal generator of a Feller semigroup on C(Iˉ)C(\bar I)C(Iˉ). The conclusion includes the semigroup law, contraction, strong right continuity at zero, positivity, preservation of the constant function 111, and a biconditional identifying the complete generator graph rather than merely an included core or its closure.

Significance

The theorem translates local coefficients into a global Markov evolution while retaining every possible finite or infinite endpoint type. The endpoint tests determine which boundary behavior is available, and the regular-boundary parameter records absorbing, reflecting, and intermediate behavior in one equation. Without the classification, writing down the differential expression does not specify a unique Feller generator because distinct domains can encode different stochastic behavior at the same endpoint.

The formal contribution is a machine-checkable statement layer for the source theorem. It makes the compactified state space, improper endpoint integrals, one-sided endpoint filters, and exact generator graph explicit. The theorem itself remains an open proof obligation in this statement-only mission. The definitions can also support later formal work on hitting distributions, killed or reflected diffusions, and comparison of boundary regimes.

Where the difficulty lies

The main difficulty is that the classification cannot be reduced to ordinary finite integrals or to boundary values at finite real points. Infinite endpoints and divergent scale or speed tests carry mathematical information; replacing extended-real integrals by totalized real integrals would collapse boundary classes. A second difficulty is identifying the full closed generator domain. Showing that the differential expression is meaningful on smooth interior functions is insufficient: the proof must connect its endpoint asymptotics to positivity, conservativity, strong continuity, and exact generation on the uniform-norm space.

Formalization scope and conventions

Endpoints use EReal, and the state space is Set.Icc r₀ r₁ in the extended line. Coefficients are constrained only on the open real interior; no boundedness or uniform ellipticity is added. The drift primitive and scale/speed tests use oriented interval integrals internally and ENNReal endpoint integrals externally, retaining +∞+\infty+∞. Absolute values correct the orientation on the left of the reference point.

The graph stores continuous endpoint extensions of both fff and GfGfGf. Interior C2C^2C2 regularity is expressed by ContDiffOn ℝ 2, and endpoint limits use the one-sided neighborhood filter induced from the extended interval. The regular parameter is required to lie in [0,1][0,1][0,1] only when both boundary tests are finite. The mission does not replace the four cases by an opaque classifier, assume finite endpoints, add a pathwise stochastic differential equation, or weaken exact generation to graph inclusion. The shared book definition of a strongly continuous contraction semigroup is reused from the earlier semigroup-generation mission rather than redeclared.

Selected references

  • Stewart N. Ethier and Thomas G. Kurtz, Markov Processes: Characterization and Convergence, Wiley, 1986, Chapter 8, Section 1, Theorem 1.1 and equations (1.1)–(1.11), printed pp. 366–367. Wiley DOI
  • Stewart N. Ethier and Thomas G. Kurtz, same volume, Chapter 4, printed p. 166, for the Feller semigroup convention used in the conclusion. Wiley DOI
5 thms1 active userReviewed
Functional Analysis·Captain: mikedeng1

Markov Processes: Characterization and Convergence 13: Whole-space diffusion generatorsTextbook

Why whole-space diffusion generators matter

Diffusion processes connect stochastic differential equations, partial differential equations, and Markov semigroups. On Euclidean space, their local behavior is encoded by a second-order operator built from a covariance field and a drift field. A central question is whether those local coefficients determine an actual Markov evolution, and whether the resulting evolution is unique. Ethier and Kurtz organize these questions through generator graphs and martingale problems, which allow the same language to cover smooth elliptic diffusions, degenerate models, and coefficients that vary measurably with time. This mission formalizes three complementary whole-space regimes from Chapter 8 of Stewart N. Ethier and Thomas G. Kurtz, Markov Processes: Characterization and Convergence (Wiley, 1986), principally Theorems 1.6, 1.7, and 2.5.

The setting

The state space is the finite-dimensional Euclidean space Rd\mathbb R^dRd, represented in Lean as EuclideanSpace ℝ (Fin d), with d>0d>0d>0. A covariance operator a(t,x)a(t,x)a(t,x) is a continuous linear endomorphism of this space, while a drift b(t,x)b(t,x)b(t,x) is a vector. For a smooth scalar test function fff, the diffusion operator is

Gtf(x)=12∑i,jaij(t,x) ∂ijf(x)+Df(x)[b(t,x)].G_t f(x)=\frac12\sum_{i,j} a_{ij}(t,x)\,\partial_{ij}f(x) + Df(x)[b(t,x)].Gt​f(x)=21​i,j∑​aij​(t,x)∂ij​f(x)+Df(x)[b(t,x)].

The factor 1/21/21/2 multiplies only the covariance-weighted second-order term. The formalization uses Fréchet derivatives evaluated in the standard coordinate directions. In the time-homogeneous cases, the smooth compactly supported graph consists of pairs (f,Gf)(f,Gf)(f,Gf) viewed in C0(Rd)×C0(Rd)C_0(\mathbb R^d)\times C_0(\mathbb R^d)C0​(Rd)×C0​(Rd), and its uniform closure is the candidate generator.

For time-dependent coefficients, a measurable process solves the time-inhomogeneous martingale problem when its initial pushforward law is prescribed and the integrated generator identity holds against every smooth compactly supported spatial test and every bounded continuous finite-history test. This formulation records the natural-past moment identities without imposing continuity of sample paths as an extra hypothesis.

Formalization targets

Goal: measurable-drift nonautonomous well-posedness

The main target is Chapter 8, Section 1, Theorem 1.7. The covariance and drift are jointly Borel measurable and locally bounded. The covariance is symmetric and nonnegative. At each fixed spatial point it is uniformly elliptic over every bounded positive time interval, and its spatial continuity is uniform over that interval. A common quadratic bound controls the covariance norm and the one-sided radial drift quantity ⟨x,b(t,x)⟩\langle x,b(t,x)\rangle⟨x,b(t,x)⟩.

For every initial probability measure, the conclusion asserts existence of a measurable-process solution on some probability space and uniqueness of all finite-dimensional distributions among every such solution on any probability space. The drift is not assumed continuous, and no stronger norm-growth condition on the drift or moment condition on the initial law is inserted.

Related target: uniformly elliptic Feller generation

Chapter 8, Section 1, Theorem 1.6 treats bounded Hölder continuous time-homogeneous coefficients with symmetric, globally uniformly elliptic covariance. It concludes that the closure of the smooth compactly supported graph is single-valued and is exactly the generator of a positive, strongly continuous contraction semigroup on C0(Rd)C_0(\mathbb R^d)C0​(Rd). Conservativity is retained through membership of the constant graph pair (1,0)(1,0)(1,0) in the bounded-pointwise closure.

Related target: degenerate smooth-core generation

Chapter 8, Section 2, Theorem 2.5 permits degenerate and unbounded covariance fields. Its assumptions are symmetry, nonnegativity, twice continuous differentiability of the matrix entries, bounded second partial derivatives, and a globally Lipschitz drift. Its conclusion is the same full Feller generation and conservativity statement. This regime is related to, but not implied by, the bounded uniformly elliptic regime.

Significance

Together the three statements separate distinct mechanisms for obtaining Markov dynamics from local diffusion data. The two autonomous theorems identify when a concrete smooth core closes to a Feller generator, including positivity, contraction, strong continuity, and conservativity. The nonautonomous theorem establishes existence and uniqueness in distribution under substantially rougher temporal behavior and merely measurable drift. Keeping the regimes together exposes the common operator while preserving their incompatible regularity hypotheses.

The formal contribution is a machine-checkable statement layer for these source results and their expression-essential definitions. It does not claim proofs of the theorems. The definitions make explicit the coordinate form of the generator, the C0C_0C0​ graph, the bounded-pointwise closure convention, and the complete finite-history martingale identities. These components can support later formal proofs and other diffusion or martingale-problem developments.

Where the difficulty lies

The source conclusions are not consequences of a single elementary continuity argument. In the Feller cases, identifying the closure of a graph with the generator requires simultaneous control of the operator domain, the semigroup properties, positivity, and conservativity. In the time-dependent case, measurability of the drift is deliberately weaker than continuity, while uniqueness must cover all measurable-process realizations and all finite collections of observation times. Replacing that conclusion by uniqueness only among continuous processes, or strengthening the drift hypotheses until a standard smooth theory applies, would change the theorem.

Formalization scope and conventions

The development uses nonnegative real time and finite-dimensional real Euclidean space. Covariance matrices act as continuous linear maps; their entries are recovered in the standard orthonormal basis. Smoothness is expressed with ContDiff, compact support with HasCompactSupport, and autonomous generator graphs in C₀. The bounded-pointwise closure is the smallest set closed under uniformly bounded pointwise sequential limits, not merely the set of one-step limits.

The martingale-problem predicate requires joint measurability of the process, equality of the initial pushforward measure, and all bounded continuous finite-history identities. Empty histories and repeated observation times are included. The goal compares the complete finite-dimensional laws of any two solutions. No vacuous solution predicate, arbitrary extension of the generator, chosen matrix square root, hidden path-continuity assumption, or proof-only theorem dependency is used.

Selected references

  • Stewart N. Ethier and Thomas G. Kurtz, Markov Processes: Characterization and Convergence, Wiley, 1986, Chapter 8, Theorems 1.6, 1.7, and 2.5. Wiley DOI
  • Stewart N. Ethier and Thomas G. Kurtz, same volume, Chapter 4 for martingale-problem well-posedness and Feller conventions, and Appendix 3 for bounded-pointwise closure. Wiley DOI
7 thms1 active userReviewed
PreviousPage 22 of 23Next

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paper
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me