Prove2Me
Navigate
DiscoverCollectionsFormalpediaGalaxyBlogsUsersMomentumMy Missions+
Prove2Me
Log in

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

Dynamic Programming

347 missions · 248 completed

Missions

Open99Completed248All347
Control TheoryMarkov ChainStochastic Systems·Captain: mikedeng1

Strong and Weak Equilibria for Time-Inconsistent Stochastic Control in Continuous Time 4: Limits of Equilibria of the Time-Discretized Problems Are Weak Equilibria in Continuous TimeResearch Paper

A decision maker is time-inconsistent when the plan that is optimal today stops being optimal tomorrow. The standard source is non-exponential discounting: a reward ttt units of time ahead is weighted by δ(t)\delta(t)δ(t), and unless δ\deltaδ is exponential the relative weight of two future dates changes as time passes. Hyperbolic and pseudo-exponential discount…

8 thms3 active usersReviewed
🏆Completed
Convex OptimizationOperations ResearchOptimization+1·Captain: mikedeng1

On the Convergence of Decomposition Methods for Multistage Stochastic Convex Programs 2: Under τ-Admissible Random Node Selection, Cuts Become Exact Almost Surely at Every Tree NodeResearch Paper

Stochastic dual dynamic programming (SDDP) and its relatives are the standard algorithms for multistage stochastic programs: hydro-thermal scheduling, energy storage and long-horizon capacity planning are solved with them in industry. They approximate each future cost function from below by a maximum of affine cuts, compute new cuts only at a few randomly…

16 thms3 active usersReviewed
🏆Completed
Linear OptimizationOperations ResearchOptimization·Captain: mikedeng1

On Solving Multistage Stochastic Programs with Coherent Risk Measures: The Inner-Approximation Policy's Nested Risk-Adjusted Cost Is at Most the Upper Bound yResearch Paper

Multistage stochastic linear programs model decisions taken over time under uncertainty: reservoir releases in hydro-thermal scheduling, production and inventory plans, capacity expansion. Their standard solution method, stochastic dual dynamic programming (SDDP, Pereira and Pinto 1991), approximates each stage's cost-to-go function from below by cutting…

8 thms3 active usersReviewed
🏆Completed
Convex OptimizationOperations ResearchOptimization·Captain: mikedeng1

On the Convergence of Decomposition Methods for Multistage Stochastic Convex Programs 1: Deterministic Cutting-Plane Bellman Approximations Become Exact Along the Generated TrajectoriesResearch Paper

Multistage stochastic programs model sequential decisions under uncertainty: hydro-thermal scheduling, inventory and capacity planning, portfolio management. The dominant solution methods, Stochastic Dual Dynamic Programming (SDDP, Pereira and Pinto 1991) and its relatives CUPPS and DOASA, never compute the Bellman (future-cost) functions of the problem.…

12 thms3 active usersReviewed
🏆Completed
Operations ResearchProbability·Captain: mikedeng1

The Bayesian Prophet: A Low-Regret Framework for Online Decision Making: Constant Regret of the Fluid Bayes Selector for the Multi-Secretary ProblemResearch Paper

In online allocation, requests arrive one at a time and each must be accepted or rejected on arrival, using a limited budget of resources. Revenue management, ad allocation and hiring all fit this pattern. A natural benchmark for an online policy is the prophet, or Offline: the decision maker who sees the whole arrival sequence in advance and acts optimally…

9 thms3 active usersReviewed
🏆Completed
Convex OptimizationOperations ResearchOptimization·Captain: mikedeng1

Robust Markov Decision Processes 2: The Affine Reward-to-Go Semidefinite Program Is Exact for One Quadratic Constraint and Conservative for SeveralResearch Paper

A Markov decision process (MDP) evaluates a policy by its expected discounted reward, which requires the transition probabilities. In practice these are estimated from data, and the optimal policy of the estimated model can perform badly under the true kernel. Robust MDPs replace the kernel by an ambiguity set of kernels and judge a policy by its worst-case…

17 thms3 active usersReviewed
🏆Completed
Operations ResearchOptimization·Captain: mikedeng1

Dynamic Type Matching: Under the Modified Monge Condition There Is an Optimal Matching Policy That Respects the Priority Relation ≻MsResearch Paper

Many platforms match two sides of a market over time: ride-hailing platforms match riders with drivers, blood banks match donated units with patients, organ allocation matches donors with recipients, and retailers pool inventory across fulfilment centres to serve orders from several regions. Demand and supply arrive in several types (locations, blood…

11 thms3 active usersReviewed
🏆Completed
Operations ResearchOptimization·Captain: mikedeng1

Dynamic Pricing with Loss-Averse Consumers and Peak-End Anchoring: Every Optimal Price Path Converges Monotonically to a Steady State Set by the Initial Minimum PriceResearch Paper

Firms that change prices over time face customers who remember past prices. A large empirical literature in marketing documents reference price effects: demand depends not only on the current price but on how that price compares with a price the customer expects, and losses (prices above the reference) weigh more than gains of the same size, the loss…

14 thms3 active usersReviewed
🏆Completed
OptimizationReinforcement Learning·Captain: mikedeng1

Global Optimality Guarantees for Policy Gradient Methods 2: Under Closure, a (c, µ)-Gradient-Dominated Policy Iteration Objective Makes ℓ (κc/(1−γ), κµ/(1−γ))-Gradient DominatedResearch Paper

Policy gradient methods optimize a parameterized control policy by gradient descent on its long-run cost. They underlie much of modern reinforcement learning, from REINFORCE and actor-critic methods to trust-region and proximal policy optimization, and they are used in operations problems such as inventory control and optimal stopping. Their objective is…

8 thms3 active usersReviewed
Markov ChainOperations ResearchProbability+1·Captain: mikedeng1

More Risk-Sensitive Markov Decision Processes 5: Under Positive Harris Recurrence a Risk-Neutral Optimal Stationary Policy Is Optimal for Power-Utility Average CostResearch Paper

Classical Markov decision theory minimizes expected cost. A decision maker who dislikes variability, such as an insurer, a portfolio manager or an operator of a service system, may instead evaluate a random cost YYY by its certainty equivalent U−1(E[U(Y)])U^{-1}(\mathbb E[U(Y)])U−1(E[U(Y)]) for an increasing utility (disutility) function UUU. Bäuerle and Rieder (More Risk-Sensitive…

9 thms3 active usersReviewed
Markov ChainOperations ResearchOptimization·Captain: mikedeng1

Optimality of (s, S) Policies in Inventory Models with Markovian Demand II: In the Nonstationary Discounted Infinite-Horizon Problem, a State-Dependent (s, S) Policy Is OptimalResearch Paper

Periodic-review inventory models with a fixed ordering cost are the setting in which the (s,S)(s,S)(s,S) policy is optimal: when the stock falls below a reorder point sss, order up to a level SSS, otherwise order nothing. Scarf (1960) proved this for independent demands over a finite horizon by introducing KKK-convexity, and Iglehart (1963) extended it to the discounted…

11 thms3 active usersReviewed
🏆Completed
Machine LearningReinforcement Learning·Captain: mikedeng1

On the Sample Complexity of Reinforcement Learning III: Non-Stationary Approximate Policy Iteration Is Within 2T Times the Regression Error Averaged over Any Competitor's State-Time DistributionTextbook

Approximate dynamic programming replaces the exact value functions of a Markov decision process by approximations, typically produced by a regression algorithm. The classical guarantees for the resulting policies are stated in the max norm: if every state-action value is estimated to within ε\varepsilonε uniformly over the state space, the greedy policy loses…

19 thms3 active usersReviewed
🏆Completed
Machine LearningOptimizationReinforcement Learning·Captain: mikedeng1

On the Sample Complexity of Reinforcement Learning V: Exact Conservative Policy Iteration Improves at Every Update and Halts within 8H²/ε² Updates with Future Advantages at Most ε/HTextbook

Policy iteration replaces the current policy by the greedy policy with respect to its state-action values. With exact values this improves the policy at every state. When the improvement step is only approximate, or is restricted to a class of policies, a greedy replacement can make the policy worse, and the resulting error bounds are stated in max norm…

18 thms3 active usersReviewed
🏆Completed
Operations ResearchOptimizationReinforcement Learning·Captain: mikedeng1

Bounded-parameter Markov Decision Processes 2: Interval Policy Evaluation Converges to the Interval Value Function of Every PolicyResearch Paper

A Markov decision process (MDP) is evaluated by solving a linear fixed-point equation built from its transition probabilities. In practice those probabilities are rarely known exactly: they are estimated from data, elicited from experts, or obtained by aggregating the states of a larger model, and each estimate comes with an error band. Givan, Leach and…

10 thms3 active usersReviewed
🏆Completed
Operations ResearchProbability·Captain: mikedeng1

Old and New Methods for Lost-Sales Inventory Systems 2: The Optimal Discounted Cost Is Nondecreasing in the Lead Time, and System L − 1 Bounds System L Through (6)Research Paper

In a periodic-review inventory system, an order placed today arrives only after a lead time of LLL periods. When unmet demand is lost rather than backlogged, the optimal ordering problem is a dynamic program whose state records the stock on hand and every order still in the pipeline. The problem goes back to Karlin and Scarf (1958) and Morton (1969, 1971). It…

5 thms3 active usersReviewed
🏆Completed
Operations ResearchOptimizationProbability·Captain: mikedeng1

Negative Dynamic Programming III: If the Actions Are Essentially Finite, Value Iteration from Zero Converges to the Optimal Return and a Stationary Policy Is OptimalResearch Paper

A dynamic programming problem in Blackwell's sense describes a controller who observes the state of a system, picks an action, collects a return, and watches the system move at random to a new state, forever. Blackwell treated the discounted case, with a bounded return and a discount factor below one (Blackwell 1965), and the positive bounded case.…

14 thms3 active usersReviewed
🏆Completed
Bandit AlgorithmsOperations ResearchProbability·Captain: mikedeng1

A Short Proof of the Gittins Index Theorem: For Finite-State Semi-Markov Bandits, Every Priority Policy Ordering States by the Index Algorithm's Indices Is OptimalResearch Paper

The multi-armed bandit problem with discounted rewards asks how to allocate effort among nnn independent projects, only one of which can be worked on at a time, when the project worked on evolves randomly and the others stay frozen. It models job scheduling, research planning, search, and sequential clinical experimentation. Gittins and Jones showed in 1974…

11 thms3 active usersReviewed
🏆Completed
Operations ResearchProbability·Captain: mikedeng1

Optimal Ordering and Rationing Policies in a Nonstationary Dynamic Inventory Model with n Demand Classes III: Under Full Backlogging, Class-j Rationing Levels Ignore Lower-Class Penalties and DemandsResearch Paper

A single stock of one product often serves customers of different importance: emergency and routine orders for spare parts, contract and spot customers, high- and low-priority patients for a blood bank. When stock runs low, the question is not only how much to order but how much of the remaining stock to hand to the less important customers now, and how…

8 thms3 active usersReviewed
🏆Completed
Operations ResearchProbability·Captain: mikedeng1

Optimal Ordering and Rationing Policies in a Nonstationary Dynamic Inventory Model with n Demand Classes II: Without Backlogging, the Critical Rationing Levels Are Nondecreasing in tResearch Paper

A firm that holds a single stock of one product often faces demand from several classes of customers that differ in importance: emergency and routine orders for spare parts, contract and spot customers, high- and low-priority users of a military supply system. When stock runs low it can pay to refuse a less important demand now in order to keep stock for a…

12 thms3 active users
🏆Completed
Markov ChainOperations ResearchStochastic Systems·Captain: mikedeng1

Quality Control under Markovian Deterioration 3: Sufficient Conditions for a Three-Region Optimal Policy and for an Optimal Policy That Never InspectsResearch Paper

Inspection and revision cost money, but continued production can allow a deteriorating process to remain in a costly state. A controller therefore needs a rule that converts incomplete information about the process into an action. Ross's two-state model makes that decision precise: the process can be good or bad, the bad state persists until revision, and…

8 thms3 active usersReviewed
🏆Completed
Markov ChainOperations ResearchStochastic Systems·Captain: mikedeng1

Quality Control under Markovian Deterioration 2: Never Inspecting or Revising Is Optimal Exactly When R ≥ C/(1 − β(1 − π))Research Paper

A production process can deteriorate without an operator seeing its current condition. Producing a unit, inspecting the process, and revising the process have different immediate costs and different effects on what can be known or controlled next period. The operator therefore needs a rule based on a belief about deterioration, rather than on the hidden…

8 thms3 active usersReviewed
🏆Completed
Operations ResearchOptimization·Captain: mikedeng1

Dynamic Pricing and Inventory Control of Substitute Products 3: Optimal Dynamic Prices When the Other Variates Keep Fixed PricesResearch Paper

A retailer that sells a line of substitutable products — sizes, colours or models of one item — over a short season without replenishment must decide how to price each variate as inventory runs down. Customers substitute: when one variate is expensive or sold out, part of its demand moves to the others. Dong, Kouvelis and Tian (M&SOM 2009) model this with a …

8 thms3 active usersReviewed
🏆Completed
Linear OptimizationMarkov ChainOperations Research·Captain: mikedeng1

Linear Programming and Markov Decision Chains 2: Representatives of Average Optimal Stationary Policies Are Optimal Dual Solutions, and Every Optimal Dual Solution Gives an Average Optimal PolicyResearch Paper

A finite Markov decision process may have several recurrent classes under one policy. In that setting, its long-run reward depends on the initial state, and a single stationary distribution does not describe the whole decision problem. Hordijk and Kallenberg developed a linear program with two families of variables to handle this multichain case. Their…

13 thms3 active usersReviewed
🏆Completed
Markov ChainOperations ResearchReinforcement Learning·Captain: mikedeng1

Planning and Acting in Partially Observable Stochastic Domains: The Witness Theorem, a Set of Useful Policy Trees Is Incomplete iff Replacing One Subtree Improves on It at Some Belief StateResearch Paper

A partially observable Markov decision process (POMDP) models an agent that acts in a stochastic world whose state it cannot see directly. The agent sees only noisy observations. POMDPs are the standard model for planning under uncertainty in robotics, dialogue systems, medical decision making and machine maintenance. The exact finite-horizon theory goes…

11 thms3 active usersReviewed
🏆Completed
Computational GeometryOperations ResearchOptimization·Captain: mikedeng1

Economic Lot Sizing: An O(n log n) Algorithm That Runs in Linear Time in the Wagner-Whitin Case 1: The Threshold Rule on Efficient Periods Picks an Optimal Next Production PeriodResearch Paper

Economic lot sizing asks how to meet a known sequence of demands while deciding when to set up production and how much to carry forward. It is a basic deterministic inventory problem: a setup has a fixed cost, production has a per-unit cost, and inventory may be held for later demand. Recomputing every possible next production period in the standard…

8 thms3 active usersReviewed
PreviousPage 5 of 14Next

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paperProve2Me Team
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me