Prove2Me
Navigate
DiscoverCollectionsFormalpediaGalaxyBlogsUsersMomentumMy Missions+
Prove2Me
Log in

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

Markov Chain

295 missions · 185 completed

Missions

Open110Completed185All295
🏆Completed
ProbabilityStochastic Systems·Captain: Shuze Chen

Markov Chains and Mixing Times I: Existence and Uniqueness of the Stationary DistributionTextbook

Finite Markov chains are the basic model for memoryless random dynamics: card shuffles, random walks on graphs and groups, Monte Carlo samplers, and queueing systems are all chains on a finite state space. The single most used fact about them is that an irreducible chain has exactly one stationary distribution — a probability vector π\piπ with π=πP\pi = \pi Pπ=πP —…

20 thms4 active usersReviewed
🏆Completed
Operations ResearchOptimizationStochastic Systems·Captain: mikedeng1

Pricing and Optimization in Shared Vehicle Systems: An Approximation Framework: Elevated-Flow-Relaxation Prices Earn an m/(m+n−1) Fraction of the Optimal State-Dependent ObjectiveResearch Paper

Bike-sharing and car-sharing systems, and ride-hailing platforms, all move a fixed fleet of units between stations. Every ride takes a unit away from its origin and leaves it at its destination, so demand that is unbalanced across stations drains some stations and floods others. Pricing is the main lever an operator has: raising the price of rides out of a…

15 thms3 active usersReviewed
🏆Completed
Operations ResearchProbabilityStochastic Systems·Captain: mikedeng1

Stability, Memory, and Messaging Trade-Offs in Heterogeneous Service Systems: Dispatching to One Remembered Server, Updated by Queue-Length Messages, Is Stable for Every Vector of Server RatesResearch Paper

A central dispatcher that routes jobs to parallel servers is the basic model behind web server farms, cloud schedulers and call centers. When the servers have different speeds, and the speeds are not known to the dispatcher, two classical policies behave very differently. Random routing is cheap but unstable as soon as some server is slower than the average…

8 thms3 active usersReviewed
🏆Completed
Operations ResearchProbabilityStochastic Systems·Captain: Shuze Chen

Introduction to Matrix Analytic Methods in Stochastic Modeling XII: Level-Dependent QBDs and Their Minimal SolutionsTextbook

Many queueing and reliability models are Markov chains on a two-dimensional state space whose first coordinate, the level, moves by at most one unit per transition, while the second, the phase, records an auxiliary state such as the phase of a service time or the number of busy servers. When the transition probabilities do not depend on the level, such a…

9 thms3 active usersReviewed
🏆Completed
Algorithmic Game TheoryOperations Research·Captain: mikedeng1

Shapley Meets Uniform: An Axiomatic Framework for Attribution in Online Advertising: In the Markov Chain Model, CASV Equals Unique-Uniform Attribution Minus Its CounterfactualResearch Paper

An advertiser observes customer paths through a sequence of states and advertising actions, then wants to assign credit for a purchase to the actions that occurred. Giving the last action all the credit misses earlier interactions; dividing credit by the number of visits can overreward a repeated action. Credit should also reflect what would have happened…

9 thms3 active usersReviewed
🏆Completed
Operations ResearchProbability·Captain: mikedeng1

A Token-Based Central Queue with Order-Independent Service Rates: Under the Assignment and Order-Independent Conditions the Stationary Distribution Has a Product FormResearch Paper

Many service systems match heterogeneous customers to heterogeneous resources: jobs to machines in a data centre, patients to beds, orders to couriers, donors to recipients. A recurring question is whether such a system, run first-come-first-served among compatible pairs, has a stationary distribution that can be written down. Two lines of work answered it…

14 thms3 active usersReviewed
🏆Completed
Operations ResearchStochastic Systems·Captain: mikedeng1

Smart Policies for Multisource Inventory Systems and General Tandem Queues with Order Tracking and Expediting: State Space and Product-Form Stationary Distribution under Policy-MResearch Paper

An inventory system can replenish demand from several stages of a supply process. When an order is placed, the manager chooses where it enters; when work finishes at one stage, the manager may send it past intermediate stages to speed delivery. These choices change the future congestion of every stage. A stationary occupancy distribution is needed to…

12 thms3 active users
🏆Completed
Machine LearningOperations ResearchReinforcement Learning·Captain: mikedeng1

A Finite Time Analysis of Temporal Difference Learning With Linear Function Approximation: Projected TD(0) Under Markov Chain Sampling Converges at Mixing-Time-Scaled RatesResearch Paper

Temporal-difference (TD) learning is the basic algorithm of reinforcement learning for estimating the value function of a fixed policy from a single stream of experience. With a linear function approximation Vθ(s)=ϕ(s)⊤θV_\theta(s)=\phi(s)^\top\thetaVθ​(s)=ϕ(s)⊤θ, TD(0) is the evaluation step inside many policy-iteration and actor–critic methods, and its behaviour governs how much…

10 thms3 active usersReviewed
🏆Completed
Operations ResearchReinforcement Learning·Captain: mikedeng1

Nonstationary Reinforcement Learning: The Blessing of (More) Optimism III: Under Uniform One-Step Reachability ζ, the Un-widened Confidence Region Contains an MDP of Diameter at Most 1/ζResearch Paper

Reinforcement learning in a changing environment has a difficulty that is absent from a fixed Markov decision process: transition data gathered earlier may describe a different system from the one the decision maker currently faces. A learning algorithm therefore estimates transitions from a recent window of observations. The estimate comes with a confidence…

8 thms3 active usersReviewed
🏆Completed
Machine LearningOperations ResearchReinforcement Learning·Captain: mikedeng1

Nonstationary Reinforcement Learning: The Blessing of (More) Optimism II: Per-Step Dynamic Regret of SWUCRL2-CW When the Widened Confidence Region Contains an MDP of Diameter at Most DResearch Paper

A reinforcement-learning agent chooses actions while learning how those actions affect future states. In a stationary model, observations collected long ago remain relevant to the current transition law. In a nonstationary Markov decision process, rewards and transition probabilities may change at each round. Old observations can then bias an estimate even…

10 thms3 active usersReviewed
🏆Completed
Linear OptimizationOperations ResearchReinforcement Learning·Captain: mikedeng1

Nonstationary Reinforcement Learning: The Blessing of (More) Optimism I: No Policy Earns More Than the Sum of Per-Step Optimal Gains Plus 4(D_max+1)√((B_r+2D_max B_p)T)Research Paper

Reinforcement learning in a changing environment needs a benchmark that remains meaningful when the transition law and rewards vary from round to round. For a stationary Markov decision process (MDP), the optimal long-run average reward is a standard benchmark. In a nonstationary MDP, summing those stationary optima across rounds is convenient, but it need…

11 thms3 active usersReviewed
🏆Completed
CombinatoricsOperations ResearchProbability·Captain: mikedeng1

Stall Economy: The Value of Mobility in Retail on Wheels I: The Magnet Rule Makes the Rendezvous Chain Doubly Stochastic, and Its Stationary Distribution Is Uniform on the Inner DiamondResearch Paper

A mobile retail stall can travel toward its next customer while that customer walks toward the stall. The customer's walking allowance changes where the stall stops, so it also changes the location from which the next trip begins. A useful trip-length calculation therefore needs a distribution for successive meeting points, not just a rule for one meeting.…

11 thms3 active usersReviewed
🏆Completed
Dynamical SystemsProbabilityStochastic Systems·Captain: mikedeng1

Coexistence and Extinction for Stochastic Kolmogorov Systems 1: If Every Boundary Ergodic Measure Repels, X Converges Exponentially Fast in Total Variation to a Unique Interior Stationary LawResearch Paper

Whether interacting species can coexist in a fluctuating environment is one of the basic questions of mathematical ecology. Deterministic models (Lotka–Volterra competition, predator–prey systems, food chains) answer it through invasion criteria: a population persists if, whenever it is rare, it grows. In a random environment the analogous criterion is…

39 thms3 active usersReviewed
🏆Completed
Optimal TransportProbability·Captain: mikedeng1

Coupling and Convergence for Hamiltonian Monte Carlo 2: Lyapunov Bounds and Local Contractivity Give W_ρ(νπⁿ, ηπⁿ) ≤ e^{−cn}W_ρ(ν, η) + βⁿλⁿ⁻¹(∫ψdν + ∫ψdη)δ(C)Research Paper

Markov chain Monte Carlo methods approximate a target probability measure by the law after nnn steps of a Markov chain. A quantitative guarantee for such a method is a bound on the distance between the law after nnn steps and the target, or between the laws of two copies of the chain started from different initial distributions. A standard route to such bounds…

7 thms3 active usersReviewed
🏆Completed
Linear algebraQuantum Information·Captain: mikedeng1

Search via Quantum Walk 2: Phase Estimation Repeated k Times on the Szegedy Walk W(P) Fixes |π⟩ and Reflects A + B ⊖ |π⟩ up to Error 2^{1−k}Research Paper

Many classical search algorithms are random walks: a Markov chain PPP on a finite state space XXX is run until it hits a marked state. Szegedy (2004) attached to every such chain a unitary quantum walk W(P)W(P)W(P), and Magniez, Nayak, Roland and Santha (SIAM J. Comput. 2011, arXiv:quant-ph/0608026) used it to search quadratically faster than the classical walk for…

11 thms3 active usersReviewed
🏆Completed
Graph Theory·Captain: marwahaha

Polynomial mixing of the switch chain for every graphical degree sequenceOpen Problem

Sampling a graph with a prescribed degree vector requires moving among realizations without changing degrees. The switch chain performs local edge replacements and asks whether these suffice for rapid global mixing. The source manuscript presents the surrounding research claim.

9 thms3 active usersReviewed
🏆Completed
CombinatoricsTheoretical Computer Science·Captain: wurtle

Approximate counting of common bases of two matroidsResearch Paper

A matroid abstracts linear independence: a finite ground set with a family of "independent" subsets that is closed under subsets and satisfies the augmentation axiom. Its maximal independent sets, the bases, all have the same size, the rank. Matroid intersection, finding a common basis of two matroids, is one of the central polynomially solvable problems of…

70 thms3 active usersReviewed
🏆Completed
Functional AnalysisProbability·Captain: mikedeng1

Logarithmic Sobolev Inequalities for Finite Markov Chains 3: Lower Bound α ≥ (1 − 2/q)λ / 2(λt_q + log M_q + (q − 2)/q) from a Bound on ‖H_{t_q}‖_{2→q}Research Paper

A finite Markov chain describes a system that changes state according to fixed transition probabilities. For a chain that settles toward an equilibrium distribution, a central question is how quickly the effect of its starting state disappears. The logarithmic Sobolev constant supplies quantitative control of this convergence and, through hypercontractive…

10 thms3 active usersReviewed
🏆Completed
Graph TheoryProbability·Captain: mikedeng1

Geometric Bounds for Eigenvalues of Markov Chains 2: Odd Closed Paths Bound the Smallest Eigenvalue, β_min ≥ −1 + 2/ιResearch Paper

The speed at which a finite reversible Markov chain approaches its stationary distribution is controlled by its eigenvalues. For a chain with eigenvalues 1=β0>β1≥⋯≥βm−1≥−11=\beta_0>\beta_1\ge\dots\ge\beta_{m-1}\ge-11=β0​>β1​≥⋯≥βm−1​≥−1, the distance to stationarity after nnn steps is governed by β∗ n\beta_*^{\,n}β∗n​, where β∗=max⁡(β1,∣βm−1∣)\beta_*=\max(\beta_1,|\beta_{m-1}|)β∗​=max(β1​,∣βm−1​∣) (Diaconis and Stroock 1991, Proposition 3).…

12 thms3 active usersReviewed
🏆Completed
Functional AnalysisProbability·Captain: mikedeng1

Logarithmic Sobolev Inequalities for Finite Markov Chains 2: Entropy Decay Ent_π(μH_t) ≤ e^{−2αt} Ent_π(μ), Improved to e^{−4αt} for Reversible ChainsResearch Paper

How fast a finite Markov chain forgets its starting point is the central quantitative question in the analysis of Monte Carlo algorithms, card shuffling and random walks on groups. The classical tool is the spectral gap λ\lambdaλ, which controls convergence in ℓ2(π)\ell^2(\pi)ℓ2(π) but loses a factor 1/π(x)1/\pi(x)1/π(x) when converted into a bound for a chain started at a point.…

14 thms3 active usersReviewed
🏆Completed
Dynamic ProgrammingOperations ResearchStochastic Systems·Captain: mikedeng1

Quality Control under Markovian Deterioration 3: Sufficient Conditions for a Three-Region Optimal Policy and for an Optimal Policy That Never InspectsResearch Paper

Inspection and revision cost money, but continued production can allow a deteriorating process to remain in a costly state. A controller therefore needs a rule that converts incomplete information about the process into an action. Ross's two-state model makes that decision precise: the process can be good or bad, the bad state persists until revision, and…

8 thms3 active usersReviewed
🏆Completed
Dynamic ProgrammingOperations ResearchStochastic Systems·Captain: mikedeng1

Quality Control under Markovian Deterioration 2: Never Inspecting or Revising Is Optimal Exactly When R ≥ C/(1 − β(1 − π))Research Paper

A production process can deteriorate without an operator seeing its current condition. Producing a unit, inspecting the process, and revising the process have different immediate costs and different effects on what can be known or controlled next period. The operator therefore needs a rule based on a belief about deterioration, rather than on the hidden…

8 thms3 active usersReviewed
🏆Completed
Operations ResearchProbabilityStochastic Systems·Captain: mikedeng1

Contact Interactions on a Lattice II: If λₖ/μ < k/(2d − 1), the Expected Size of a Contact Process Is Integrable in TimeResearch Paper

The contact process is the basic model of an infection spreading on a graph: infected sites recover at a constant rate, and healthy sites are infected at a rate that depends on how many of their neighbours are infected. T. E. Harris introduced it in Contact Interactions on a Lattice (Ann. Probab. 2, 1974, doi:10.1214/aop/1176996493), and it became one of…

13 thms3 active usersReviewed
🏆Completed
Dynamic ProgrammingLinear OptimizationOperations Research·Captain: mikedeng1

Linear Programming and Markov Decision Chains 2: Representatives of Average Optimal Stationary Policies Are Optimal Dual Solutions, and Every Optimal Dual Solution Gives an Average Optimal PolicyResearch Paper

A finite Markov decision process may have several recurrent classes under one policy. In that setting, its long-run reward depends on the initial state, and a single stationary distribution does not describe the whole decision problem. Hordijk and Kallenberg developed a linear program with two families of variables to handle this multichain case. Their…

13 thms3 active usersReviewed
🏆Completed
Dynamic ProgrammingOperations ResearchReinforcement Learning·Captain: mikedeng1

Planning and Acting in Partially Observable Stochastic Domains: The Witness Theorem, a Set of Useful Policy Trees Is Incomplete iff Replacing One Subtree Improves on It at Some Belief StateResearch Paper

A partially observable Markov decision process (POMDP) models an agent that acts in a stochastic world whose state it cannot see directly. The agent sees only noisy observations. POMDPs are the standard model for planning under uncertainty in robotics, dialogue systems, medical decision making and machine maintenance. The exact finite-horizon theory goes…

11 thms3 active usersReviewed
PreviousPage 3 of 8Next

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paperProve2Me Team
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me