Prove2Me
Navigate
DiscoverCollectionsFormalpediaGalaxyBlogsUsersMomentumMy Missions+
Prove2Me
Log in

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

Stochastic Systems

635 missions · 329 completed

The mathematics of systems that evolve under randomness, modeled as families of random variables indexed by time — from Markov chains and martingales to Brownian motion and stochastic differential equations. The field spans stochastic analysis, filtering and optimal control under uncertainty, ergodic behavior of random dynamics, and concentration of measure, with models reaching across physics, engineering, finance, and biology.

Missions

Open306Completed329All635
🏆Completed
Dynamical SystemsProbability·Captain: mikedeng1

Subadditive Ergodic Theory 2: For Every Increasing Γ There Is a C^∞ Process with Stationary Increments and P{|y_t| ≤ Γ(t) for All Large t} = 0Research Paper

Kingman's subadditive ergodic theorem (Theorem 1 of Kingman 1973, first proved in Kingman 1968) says that for a discrete-parameter subadditive process x=(xst)x=(x_{st})x=(xst​) the ratio x0t/tx_{0t}/tx0t​/t converges with probability one and in mean. Applications in first-passage percolation, products of random matrices and longest increasing subsequences often come with a…

7 thms2 active usersReviewed
🏆Completed
Dynamic ProgrammingLinear OptimizationOperations Research+2·Captain: mikedeng1

Linear Programming and Finite Markovian Control Problems IX: Discounted Semi-Markov Decision Processes — the Value Vector Is the Smallest Superharmonic VectorTextbook

A decision maker may choose an action whenever a system changes state, even though the time until the next change is random. Maintenance and replacement decisions are examples: a repair choice changes both the next operating state and the length of time before another choice is available. A discrete-time Markov decision model gives every decision epoch the…

6 thms2 active usersReviewed
Operations ResearchOptimizationProbability·Captain: mikedeng1

Mean-Variance Hedging in Continuous Time: The Feedback Futures Strategy Φ(G*) Minimizes the Expected Squared Deviation of Terminal Wealth from Any Target LevelResearch Paper

A firm that will receive or deliver a quantity of a commodity, currency or security at a future date carries price risk until that date. When the asset itself cannot be traded in the meantime, the standard instrument for reducing that risk is a futures contract on a correlated asset: the firm takes a position in futures and adjusts it over time, and the…

12 thms2 active usersReviewed
Control TheoryProbability·Captain: mikedeng1

Backward-Forward Stochastic Differential Equations 1: A Backward Equation Driven by a Bounded-Variation Integrator Has a Unique Adapted Solution in L¹(μ) Under Integrability AloneResearch Paper

Backward stochastic differential equations (BSDEs) prescribe a process by its terminal value rather than its initial value: the unknown VVV must be adapted to the information flow, yet must reach a given random variable YYY at the horizon TTT. They arise in mathematical finance (pricing and hedging, where YYY is a contingent claim), in stochastic control (adjoint…

10 thms2 active usersReviewed
🏆Completed
Operations ResearchProbabilityReinforcement Learning·Captain: mikedeng1

Learning in Structured MDPs with Convex Cost Functions: Improved Regret Bounds for Inventory Management: Base-Stock Values from Any Two Starting States Differ by at Most 36 max(h,p)LxResearch Paper

The lost-sales inventory problem with lead times is a basic model of operations management. A retailer reviews one product's stock each period and places an order that arrives LLL periods later. Demand that cannot be met from stock on hand is lost, and the retailer pays a holding cost hhh per unit left on the shelf and a penalty ppp per unit of lost demand. The…

9 thms2 active usersReviewed
🏆Completed
CombinatoricsMarkov ChainOperations Research+1·Captain: mikedeng1

Reversibility and Stochastic Networks VI: The Ewens Sampling Distribution Is Consistent Under Sampling Without ReplacementTextbook

The neutral theory of molecular evolution holds that much of the genetic variation observed at the molecular level is caused by selectively neutral mutations rather than by selection. To test it against data one needs the distribution of allele frequencies that a neutral model predicts, and in practice that distribution has to be compared with a sample from…

9 thms2 active usersReviewed
🏆Completed
Operations ResearchProbability·Captain: mikedeng1

Quantifying the Bullwhip Effect in a Simple Supply Chain: The Impact of Forecasting, Lead Times, and Information 2: Without Shared Demand Information the Bullwhip Bound Is MultiplicativeResearch Paper

The bullwhip effect is the observation that the variability of orders grows as one moves up a supply chain, from the retailer to the wholesaler, the distributor and the factory, even when customer demand is stable. It was documented in industry practice by Lee, Padmanabhan and Whang (Management Science, 1997), who identified demand forecasting as one of its…

5 thms2 active usersReviewed
🏆Completed
Markov ChainOperations ResearchProbability·Captain: mikedeng1

On the Stochastic Matrices Associated with Certain Queuing Processes 1: The M/G/1 Imbedded Chain Is Ergodic iff ρ < 1 and Recurrent iff ρ ≤ 1Research Paper

Many queues observed at well-chosen instants are Markov chains on the nonnegative integers. For the single-server queue with Poisson arrivals and general service times (M/G/1), D. G. Kendall showed in 1951 that the number of customers left behind at successive departure epochs is such a chain, the imbedded Markov chain (Kendall 1951; Kendall 1953). Whether…

9 thms2 active usersReviewed
🏆Completed
Algorithmic Game TheoryControl TheoryOperations Research·Captain: mikedeng1

Nonzero-Sum Stochastic Differential Games with Impulse Controls: A Verification Theorem with Applications 2: An Explicit Family of Nash Equilibria for the Linear Impulse GameResearch Paper

Impulse control models an agent who acts on a random system through discrete interventions, each with a fixed cost: inventory replenishment, cash management, exchange-rate interventions by a central bank. With two agents whose objectives conflict, the problem becomes a nonzero-sum stochastic differential game with impulse controls. Before the work of Aïd,…

11 thms2 active usersReviewed
🏆Completed
Markov ChainOperations ResearchProbability·Captain: mikedeng1

Optimization of Multiclass Queueing Networks: Polyhedral and Nonlinear Characterizations of Achievable Performance I: Quadratic Potential Functions Bound Mean Response Times in Open NetworksResearch Paper

Scheduling in a multiclass queueing network asks which waiting job a server should work on next when jobs of several types share stations and revisit them along fixed routes. Such networks model semiconductor wafer fabs, job shops and communication switches. Optimal policies are rarely computable: the state space is countably infinite, and even deciding…

8 thms2 active usersReviewed
Control TheoryOperations ResearchProbability·Captain: mikedeng1

Dynamic Scheduling of a System with Two Parallel Servers in Heavy Traffic with Resource Pooling: The Threshold Policy Is Asymptotically OptimalResearch Paper

Many service systems route several classes of work to servers with overlapping skills: call centers with cross-trained agents, manufacturing cells with flexible machines, computing clusters with heterogeneous processors. Choosing which server works on which class at each moment is a dynamic scheduling problem. Exact optimal policies are out of reach except…

14 thms2 active usersReviewed
🏆Completed
Dynamical SystemsOperations ResearchProbability·Captain: mikedeng1

Dynamics of Stochastic Approximation Algorithms 7: Weak Limit Points of the Occupation Measures of a Weak Asymptotic Pseudotrajectory Are InvariantResearch Paper

Stochastic approximation algorithms are recursions xn+1−xn=γn+1(F(xn)+Un+1)x_{n+1}-x_n=\gamma_{n+1}(F(x_n)+U_{n+1})xn+1​−xn​=γn+1​(F(xn​)+Un+1​) driven by small steps γn\gamma_nγn​ and noise Un+1U_{n+1}Un+1​; they include the Robbins–Monro scheme, stochastic gradient methods and learning dynamics in games. The ODE method studies their long-run behaviour by comparing a time-interpolation of the iterates with the…

10 thms2 active usersReviewed
🏆Completed
Dynamical SystemsOperations Research·Captain: mikedeng1

Dynamics of Stochastic Approximation Algorithms 1: Limit Sets of Precompact Asymptotic Pseudotrajectories Are Internally Chain TransitiveResearch Paper

A stochastic approximation algorithm updates an estimate by small noisy steps, xn+1−xn=γn+1(F(xn)+Un+1)x_{n+1}-x_n=\gamma_{n+1}(F(x_n)+U_{n+1})xn+1​−xn​=γn+1​(F(xn​)+Un+1​), with step sizes γn→0\gamma_n\to0γn​→0 and noise UnU_nUn​. Such recursions underlie stochastic gradient descent, adaptive control, reinforcement learning (Q-learning, temporal-difference methods) and learning in games (fictitious play, reinforcement…

12 thms2 active usersReviewed
🏆Completed
Dynamical SystemsOperations Research·Captain: mikedeng1

Dynamics of Stochastic Approximation Algorithms 2: Under A1 and A2 the Interpolated Process Is an Asymptotic Pseudotrajectory of the Flow of FResearch Paper

A stochastic approximation algorithm is a recursion

9 thms2 active usersReviewed
Control TheoryMechanism DesignOperations Research·Captain: mikedeng1

Optimal Electricity Demand Response Contracting with Responsiveness Incentives 1: The Producer's Second-Best Value in Closed FormResearch Paper

Demand response asks electricity consumers to lower or smooth their consumption when generation is expensive, in exchange for payments. Field trials such as Low Carbon London showed two effects of such incentives: consumers reduce their average consumption, and the variability of their response depends on how much effort they put into it. A producer who…

7 thms2 active usersReviewed
Algorithmic Game TheoryControl TheoryOperations Research·Captain: mikedeng1

Nonzero-Sum Stochastic Differential Games with Impulse Controls: A Verification Theorem with Applications 1: Regular Solutions of the Quasi-Variational Inequalities Give Nash Equilibrium PayoffsResearch Paper

Many economic and engineering systems are steered by agents who act at discrete instants rather than continuously: a central bank intervenes on an exchange rate, two energy producers adjust a shared stock, a firm rebalances inventory. Each action has a fixed cost, so continuous control is not realistic. The mathematical model is impulse control: the state…

11 thms2 active usersReviewed
🏆Completed
Markov ChainNumerical AnalysisOperations Research+1·Captain: mikedeng1

Fundamentals of Queueing Theory X: Uniformization of Continuous-Time Markov ChainsTextbook

Most Markovian queueing models have no closed-form transient solution. The M/M/1 queue already needs modified Bessel functions (Chapter 2 of the book), and a finite-capacity or multi-class model with state-dependent rates has no closed form at all. What an analyst can always write down is the system of forward equations p′(t)=p(t)Qp'(t)=p(t)Qp′(t)=p(t)Q for the state…

8 thms2 active usersReviewed
AnalysisOperations ResearchProbability·Captain: mikedeng1

Elements of Queueing Theory Ib: Ergodicity and Stochastic IntensityTextbook

Chapter 1 of Baccelli and Brémaud's Elements of Queueing Theory has two halves. The first builds Palm calculus from the Matthes definition of P⁰_N and reaches the Swiss army formula. This mission is the second: §§1.6, 1.8 and 1.9, which supply the two things the rest of the book runs on.

13 thms2 active usersReviewed
🏆Completed
Markov ChainOperations ResearchProbability·Captain: mikedeng1

Stochastic Dynamic Programming and the Control of Queueing Systems XIV: Conforming Approximating Sequences for Markov ChainsTextbook

Countable-state Markov chains are the standard model of queues with unbounded buffers, but any numerical computation of their long-run behaviour works on a finite state space. The usual remedy is truncation: restrict the chain to a finite set SNS_NSN​ and redistribute the probability of leaving SNS_NSN​ back into it. Whether the steady state probabilities and average…

10 thms2 active usersReviewed
Numerical AnalysisProbability·Captain: mikedeng1

Euler Approximations with Varying Coefficients III: Uniform Lq-Convergence with Order 1/2Research Paper

Stochastic differential equations whose coefficients grow faster than linearly (for instance stochastic volatility models such as the 3/2-model) cannot be simulated with the classical explicit Euler–Maruyama scheme: Hutzenthaler, Jentzen and Kloeden showed that its moments diverge when the drift or diffusion grows superlinearly (Proc. R. Soc. A, 2011).…

14 thms2 active usersReviewed
🏆Completed
Dynamic ProgrammingOperations ResearchProbability·Captain: mikedeng1

Computing Optimal (s, S) Inventory Policies I: The Renewal Closed Form for the Discounted Cost of a Stationary (s, S) PolicyResearch Paper

The periodic-review inventory problem with a fixed ordering cost is one of the basic models of operations research. A firm reviews its stock once per period, may order at a cost KKK per order plus a unit cost, and then faces a random demand; unmet demand is backlogged. Scarf (1960) and Iglehart (1963) showed that for this model an (s,S)(s, S)(s,S) policy is optimal:…

12 thms2 active usersReviewed
🏆Completed
Dynamic ProgrammingMarkov ChainOperations Research·Captain: mikedeng1

Markov-Renewal Programming. II: Infinite Return Models, Example II: Gain and Bias of the Infinite-Step ReturnResearch Paper

A Markov-renewal program is a sequential decision model in which a system moves among finitely many states, the time between transitions is random and may depend on both the current and the next state, and a reward accrues during each sojourn. It generalizes the Markov decision process, where every transition takes one time unit, and is the standard model…

10 thms2 active usersReviewed
🏆Completed
Dynamic ProgrammingMarkov ChainOperations Research·Captain: mikedeng1

Markov-Renewal Programming. II: Infinite Return Models, Example I: Policy Iteration Finds a Stationary Policy of Maximal Gain RateResearch Paper

Many controlled systems do not move in unit time steps. A machine runs for a random time before it breaks down, a repair takes a random time, a queue sits in a state until the next arrival or departure. A Markov-renewal program (in later terminology a semi-Markov decision process) models such a system: the sequence of visited states is a Markov chain…

7 thms2 active usersReviewed
🏆Completed
Dynamic ProgrammingOperations ResearchProbability·Captain: mikedeng1

Exit Problems for Spectrally Negative Lévy Processes and Applications to (Canadized) Russian Options III: Optimal Stopping for the Canadized Russian OptionResearch Paper

The Russian option of Shepp and Shiryaev pays its holder, at an exercise time of their choosing, the running maximum of the stock price, discounted, with no fixed maturity. It is the standard example of a perpetual lookback option, and its value is a model problem of optimal stopping for a two-dimensional Markov process (the price and its running maximum).…

18 thms2 active usersReviewed
Operations ResearchProbability·Captain: mikedeng1

Diffusion approximations for open queueing networks with service interruptions 2: jump-diffusion heavy-traffic limit for long up and down timesResearch Paper

Servers in manufacturing lines, communication links and service systems break down, are taken offline for maintenance, or go on vacation. When the interruptions are rare but long, they dominate congestion. A single down period can build a backlog that takes a long time to clear, and in a network that backlog propagates downstream. Standard heavy-traffic…

10 thms2 active usersReviewed
PreviousPage 21 of 26Next

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paperProve2Me Team
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me