Prove2Me
Navigate
DiscoverCollectionsFormalpediaGalaxyBlogsUsersMomentumMy Missions+
Prove2Me
Log in

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

Markov Chain

295 missions · 185 completed

Missions

Open110Completed185All295
Operations ResearchProbabilityStochastic Systems·Captain: mikedeng1

An Approximation to the Invariant Measure of the Limiting Diffusion of G/Ph/n + GI Queues in the Halfin–Whitt Regime and Related Asymptotics 2: CLT for Occupation Measures With Variance µ(|σ′∇f|²)Research Paper

In a many-server queue with nnn servers, the Halfin–Whitt (quality-and-efficiency-driven) regime lets n→∞n\to\inftyn→∞ while the arrival rate approaches capacity at rate n\sqrt nn​. In this regime the centred and scaled number of customers in each service phase of a G/Ph/n+GIG/Ph/n+GIG/Ph/n+GI queue (general arrivals, phase-type service, general patience) converges to a ddd-dimensional…

16 thms2 active usersReviewed
Control TheoryStochastic Systems·Captain: mikedeng1

Strong and Weak Equilibria for Time-Inconsistent Stochastic Control in Continuous Time 2: Under Pseudo-Exponential Discounting, a Two-State Generator Is a Weak but Not a Strong EquilibriumResearch Paper

An agent whose preferences change over time cannot simply optimize once: a plan that is optimal today may be abandoned tomorrow by the agent's own "future selves". This is time inconsistency, and its standard source is non-exponential discounting, studied since Strotz (1955). The usual response, going back to Strotz and made rigorous in continuous time by…

11 thms2 active usersReviewed
ProbabilityStatisticsStochastic Systems·Captain: mikedeng1

Can Local Particle Filters Beat the Curse of Dimensionality?: The Block Particle Filter Has a Local Error Bound Uniform in Time and in the Model DimensionResearch Paper

Nonlinear filtering asks for the conditional law of an unobserved Markov process given noisy partial observations of it. It is the computational core of data assimilation in weather forecasting, oceanography and geophysics, of target tracking, and of inference in large networked systems. The standard numerical method, the particle filter, replaces the…

13 thms2 active usersReviewed
Functional AnalysisProbabilityStatistics·Captain: mikedeng1

Spectral Gaps for a Metropolis–Hastings Algorithm in Infinite Dimensions I: The pCN Algorithm Has a Wasserstein Spectral Gap Uniform in the Dimension When the Potential Is Globally LipschitzResearch Paper

Sampling from a distribution described relative to a Gaussian reference law is common in Bayesian inverse problems, where an unknown function or initial condition is inferred from noisy observations. Such a distribution is naturally infinite dimensional before it is approximated on a computer. A numerical method must therefore behave reliably as the…

10 thms2 active usersReviewed
Dynamic ProgrammingOperations ResearchProbability·Captain: mikedeng1

More Risk-Sensitive Markov Decision Processes 2: The Finite-Horizon Discounted Problem Is Solved by Value Iteration on the State Extended by Accumulated Cost and DiscountResearch Paper

Sequential decisions often have costs whose timing matters. A controller may choose an action now, observe a random next state, and choose again. Ordinary expected-cost minimization averages the sum of the costs. A risk-sensitive criterion first applies an increasing function UUU to the total cost and then takes its expectation. The curvature of UUU changes how…

6 thms2 active usersReviewed
Mathematical PhysicsProbability·Captain: mikedeng1

The Endpoint Distribution of Directed Polymers 1: In the Low-Temperature Phase the Endpoint Distribution Is Asymptotically Purely Atomic, and at High Temperature It Is NotResearch Paper

A directed polymer in a random environment is a random walk whose paths are weighted by the disorder they encounter. The walk favors paths with large accumulated weight, and its endpoint distribution can concentrate on a small number of sites even though the number of possible paths grows exponentially. Understanding how much probability remains spread…

21 thms2 active usersReviewed
Operations ResearchProbabilityStochastic Systems·Captain: mikedeng1

Validity of Heavy Traffic Steady-State Approximations in Generalized Jackson Networks: Scaled Stationary Queue Lengths Converge to the Stationary Distribution of the Reflected Brownian MotionResearch Paper

Open networks of single-server queues with general (non-exponential) interarrival and service times, generalized Jackson networks (GJNs), model manufacturing lines, communication networks and service systems. Their stationary distributions are almost never available in closed form. The standard engineering approximation replaces the scaled queue-length…

18 thms2 active usersReviewed
Algorithmic Game TheoryProbability·Captain: mikedeng1

The Evolution of Conventions III: In 2 × 2 Games with Two Strict Equilibria, the Generically Stable Equilibria of Adaptive Play Are the Weakly Risk Dominant OnesResearch Paper

Many games have several strict Nash equilibria, and equilibrium analysis alone does not say which one a population will settle on. Harsanyi and Selten (1988) proposed risk dominance as a selection criterion for 2×22\times22×2 games: of two strict equilibria, select the one whose basin is "larger" in a precise payoff sense. That criterion is axiomatic. H. Peyton…

45 thms2 active usersReviewed
Dynamic ProgrammingMachine LearningProbability+1·Captain: mikedeng1

An Analysis of Temporal-Difference Learning with Function Approximation 1: Linear TD(λ) Converges with Probability 1 to the Fixed Point of ΠT^(λ), with Error ≤ ‖ΠJ* − J*‖_D/(1 − α(1−λ)/(1−αλ))Research Paper

Temporal-difference learning, TD(λ\lambdaλ), is the basic algorithm of reinforcement learning for estimating the expected discounted cost of a Markov chain from observed transitions (Sutton, 1988). On large or infinite state spaces it is run with a linear function approximator: the cost-to-go is represented as a weighted sum of KKK fixed basis functions, and…

13 thms2 active usersReviewed
Dynamic ProgrammingOperations ResearchOptimization·Captain: mikedeng1

Inventory Control in a Fluctuating Demand Environment II: With a Fixed Order Cost, a World-Dependent (r, S) Policy Is Optimal and Lies within Veinott-Type BoundsResearch Paper

Demand for many products is not stationary: it rises and falls with the economy, the season, a product's life cycle or a customer's state. Song and Zipkin (1993) (DOI 10.1287/opre.41.2.351) model this by a world-driven demand process: an exogenous Markov chain AAA (the "state of the world") whose current state sets the rate of a Poisson demand stream. The…

17 thms2 active usersReviewed
ProbabilityStochastic Systems·Captain: mikedeng1

Random Walks in a Random Environment 3: With One-Way Mirrors σ ∈ {0, θ}, X_n/(n/log_θ n) → ½ in Probability if γθ = 1, and n^(−1/ρ)X_n Has Limit Laws Along Subsequences if γθ > 1Research Paper

A random walk in a random environment uses transition probabilities selected once for each site, rather than probabilities resampled at every step. Even when the site environments are independent and identically distributed, the walker revisits the same local bias whenever it returns to a site. Its long-time motion can therefore differ sharply from that of…

9 thms2 active usersReviewed
CombinatoricsProbability·Captain: mikedeng1

Coalescents With Multiple Collisions 1: A Coalescent on Partitions of ℕ With k-Fold Merger Rates λ_{b,k} Exists Iff λ_{b,k} = ∫ x^{k−2}(1−x)^{b−k} Λ(dx)Research Paper

A coalescent models a collection of ancestral lineages merging as time runs backward. If every event joins exactly two lineages, a single pairwise merger rate describes the finite system. Pitman’s 1999 paper asks which rates permit an event to join any number of blocks, while still giving one process on partitions of all positive integers. This matters when…

7 thms2 active usersReviewed
Graph TheoryProbability·Captain: mikedeng1

The Evolution of Conventions II: The Stochastically Stable States of a Regular Perturbation of a Finite Markov Chain Are the Recurrent Classes of Minimum Stochastic PotentialResearch Paper

Many models in economics, biology and computer science are finite Markov chains that have several absorbing states or recurrent classes, so their long-run behaviour depends on where they start. Adding a small amount of noise (mistakes, mutations, experimentation) makes the chain irreducible, and the noisy chain then has a unique stationary distribution. As…

11 thms2 active usersReviewed
Dynamic ProgrammingOperations ResearchProbability·Captain: mikedeng1

Inventory Control in a Fluctuating Demand Environment III: Under Stochastic Monotonicity, the Myopic and Optimal Basestock Levels Are Nondecreasing in the World StateResearch Paper

Demand for many products does not arrive at a constant rate. It moves with an underlying environment: the economy, the season, a product's life-cycle stage, the state of a customer's own operations. Song and Zipkin (Oper. Res. 41(2):351–370, 1993) modelled this environment as a continuous-time Markov chain, the world, whose current state sets the Poisson…

14 thms2 active usersReviewed
Mathematical PhysicsQuantum Information·Captain: iris

Rapid mixing of quantum spin chains at any finite temperatureResearch Paper

Thermal (Gibbs) states describe quantum systems in equilibrium, and preparing them on a quantum computer is a basic task for quantum simulation. A common route is a quantum Gibbs sampler: a Lindbladian whose dynamics converges to the Gibbs state σβ=e−βH/tr⁡e−βH\sigma_\beta=e^{-\beta H}/\operatorname{tr}e^{-\beta H}σβ​=e−βH/tre−βH from any initial state, the quantum analogue of a Markov…

27 thms1 active userReviewed
CombinatoricsProbabilityTheoretical Computer Science·Captain: mikedeng1

Tight Variable-Occurrence Bounds and Exponential Separations for Moser–Tardos and WalkSAT on 3-CNFResearch Paper

WalkSAT and the Moser–Tardos algorithm (MT) are the two standard clause-driven local-repair procedures for satisfiability. Both keep a complete assignment and repeatedly pick an unsatisfied clause. WalkSAT flips one variable of it (Papadimitriou 1991, Schöning 1999); MT resamples all of its variables with fresh random bits, which is the algorithmic form of…

28 thms1 active userReviewed
Computational GeometryProbabilityTheoretical Computer Science·Captain: Shuze Chen

An FPRAS for Mixed VolumesResearch Paper

For compact convex sets K1,…,Km⊆RnK_1,\dots,K_m\subseteq\mathbb{R}^nK1​,…,Km​⊆Rn, Minkowski showed that the volume of a weighted Minkowski sum is a homogeneous polynomial of degree nnn in the weights,

27 thms1 active userReviewed
cryptographyProbability·Captain: mikedeng1

Comparison is All You Need: Settling the ttt-wise independence of AESResearch Paper

A block cipher maps fixed-length plaintext blocks to ciphertext blocks using a secret key. One way to study its statistical behavior is to fix several distinct plaintexts and ask whether their ciphertexts resemble the outputs of a uniformly random permutation. This is approximate t-wise independence: it controls every test of the joint outputs at those…

21 thms1 active userReviewed
Probability·Captain: mikedeng1

Learning AI Agents, Market Makers, and Stress in Automated MarketsResearch Paper

Two markets can exhibit the same distribution of individual returns while arranging losses differently in time. For a monitor concerned with consecutive losses, the duration of a stressed interval is a separate object from the frequency of stressed periods. The question here is whether this distinction survives a model in which trades respond to the…

5 thms1 active userReviewed
Complexity TheoryProbabilityTheoretical Computer Science·Captain: iris

Efficient Stable Word-RAM Simulation by Population ProtocolsResearch Paper

Population protocols model computation by many identical, anonymous agents with small memory that interact in randomly chosen pairs: chemical reaction networks, sensor swarms, molecular programming AAD+06. Inputs and outputs are unary counts of agents in designated states. With a constant number of states, the predicates that can be computed stably (correctl…

31 thms1 active userReviewed
ProbabilityStochastic SystemsTheoretical Computer Science·Captain: iris

Fast Minority Dynamics with Θ(log⁡2n)\Theta(\log^2 n)Θ(log2n)-Size SamplesResearch Paper

Binary consensus asks nnn agents, each holding opinion 000 or 111, to agree on one value by exchanging information. In the synchronous kkk-Pull model every agent observes, in each round, the opinions of kkk agents chosen independently and uniformly at random, and updates its opinion by a fixed local rule. For the MAJORITY rule, where an agent adopts the majority of…

30 thms1 active userReviewed
ProbabilityStochastic SystemsTheoretical Computer Science·Captain: iris

Minority Dynamics with Small SamplesResearch Paper

Consensus and bit dissemination are basic tasks for populations of simple agents that interact through random samples: biological collectives, chemical reaction networks and gossip protocols in distributed systems. In consensus, a population holding two opinions must come to agree on one of them. In bit dissemination, one or more sources hold a fixed…

23 thms1 active userReviewed
Operations ResearchProbabilityReinforcement Learning·Captain: iris

A Policy Gradient Algorithm for the Risk-Sensitive Exponential Cost MDP: The Trajectory-Based Algorithm Drives the Truncated and Untruncated Cost Gradients to Zero Almost SurelyResearch Paper

The risk-sensitive exponential cost of a Markov chain replaces the long-run average of the one-step cost by the exponential growth rate of the expected exponential of the accumulated cost. It penalizes rare sample paths with high cost, and it goes back to Howard and Matheson (1972). For a finite chain the growth rate is the logarithm of a Perron eigenvalue (…

19 thms1 active userReviewed
Operations ResearchProbabilityStochastic Systems·Captain: Shuze Chen

Stability of Queueing Networks III: Fluid-Limit Stability Implies Positive Harris RecurrenceTextbook

A queueing network can have spare capacity at every station and still accumulate an ever-growing population of jobs. Service priorities and routing can prevent the available capacity from being used where it is needed. For networks with general interarrival and service distributions, an explicit stationary distribution is usually unavailable. The stability…

17 thms1 active userReviewed
Operations ResearchProbabilityStochastic Systems·Captain: Shuze Chen

Stability of Queueing Networks I: Product-Form Stationary DistributionsTextbook

A queueing network describes jobs that receive service at several stations and move between them before leaving. A stationary distribution answers questions about the network at a typical time: how many jobs are waiting, which classes they belong to, and how jobs are arranged at each station. Such distributions can be difficult to determine because a…

9 thms1 active userReviewed
PreviousPage 4 of 5Next

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paperProve2Me Team
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me