Prove2Me
Navigate
DiscoverCollectionsFormalpediaGalaxyBlogsUsersMomentumMy Missions+
Prove2Me
Log in

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

Bandit Algorithms

215 missions · 122 completed

Missions

Open93Completed122All215
🏆Completed
Algorithmic Game TheoryConvex OptimizationMachine Learning·Captain: iris

Regret Bounds for Profit Maximization in Bilateral Trade: Learning the Best Posted-Price Mechanism: One-Bit Feedback Achieves O(Tlog⁡5/2T)O(\sqrt{T}\log^{5/2}T)O(T​log5/2T) Regret for Independent MHR ValuationsResearch Paper

A broker stands between a seller who owns one item and a buyer who may want it. In every round t=1,…,Tt=1,\dots,Tt=1,…,T a fresh pair of private valuations (st,bt)∈[0,1]2(s_t,b_t)\in[0,1]^2(st​,bt​)∈[0,1]2 arrives, and the broker posts a price ptp_tpt​ to the seller and a price qtq_tqt​ to the buyer. Trade happens when both accept (st≤pts_t\le p_tst​≤pt​ and qt≤btq_t\le b_tqt​≤bt​), and the broker earns qt−ptq_t-p_tqt​−pt​. The broker sees only…

21 thms4 active usersReviewed
🏆Completed
Algorithmic Game TheoryMachine Learning·Captain: iris

Regret Bounds for Profit Maximization in Bilateral Trade: Learning the Best Posted-Price Mechanism: Every Algorithm Has Ω(T3/5)\Omega(T^{3/5})Ω(T3/5) Regret on Some Independent Regular ValuationsResearch Paper

A broker stands between a seller and a buyer with private valuations. It posts a price ppp to the seller and a price qqq to the buyer; trade happens if the seller accepts ppp and the buyer accepts qqq, and the broker keeps the spread q−pq-pq−p. When such trades repeat, the broker must learn good prices from experience. Brüstle, di Gregorio, Fusco and Leonardi determine…

14 thms4 active usersReviewed
Machine LearningProbabilityReinforcement Learning·Captain: iris

Introducing Feedback Delays to Non-Delayed Adversarial MDPs: Optimal √S and √K Dependencies in Tabular and Linear AMDPsResearch Paper

An episodic adversarial Markov decision process (AMDP) models a learner who repeats an HHH-step task for KKK episodes while an adversary fixes the loss function of every episode in advance. The learner sees only the losses of the state-action pairs it visits (bandit feedback), and it does not know the transition law. The standard measure of performance is the re…

82 thms4 active usersReviewed
🏆Completed
Machine Learning·Captain: iris

Best-of-Both-Worlds FTRL for Linear Bandits: The Interplay of Information Acquisition, Corruption, and GeometryResearch Paper

In a linear bandit a learner repeatedly picks an action AtA_tAt​ from a known set A⊂Rd\mathcal A\subset\mathbb R^dA⊂Rd and observes only a noisy scalar loss YtY_tYt​ with mean At⊤θtA_t^\top\theta_tAt⊤​θt​. Linear bandits model sequential decisions with structured feedback: routing, recommendation, pricing, and experimental design. Two benchmarks are classical. Against an adversarial envi…

68 thms4 active usersReviewed
Machine LearningOperations ResearchProbability·Captain: iris

No-Regret Revenue Maximization in Sequential Market Making Under Inventory ConstraintsResearch Paper

A market maker quotes two prices for an asset: a bid, at which it buys, and an ask, at which it sells. It earns the spread when both sides trade, and it carries inventory risk when they do not balance: a maker that buys much more than it sells is exposed to the price of the asset itself, which defeats the purpose of quoting. Classical market-making models…

29 thms4 active usersReviewed
Machine LearningOperations ResearchStatistics·Captain: iris

Dynamic Batch Learning in High-Dimensional Sparse Linear Contextual Bandits: LASSO Batch Greedy Learning Has Regret at Most O~(M3/2Ts0(T/s0)1/(2(2M−1))/(γρ))\tilde O(M^{3/2}\sqrt{Ts_0}(T/s_0)^{1/(2(2^M-1))}/(\gamma\rho))O~(M3/2Ts0​​(T/s0​)1/(2(2M−1))/(γρ))Research Paper

Personalized medicine, online advertising and news recommendation all choose among a few actions using a very large number of features per decision, of which only a few matter. They also rarely update a policy after every decision: clinical trials report in stages, and production recommenders are retrained in batches. Dynamic batch learning models this…

44 thms4 active usersReviewed
Machine LearningOperations ResearchProbability·Captain: iris

Technical Note—On Dynamic Pricing with Covariates: UCB Pricing Has Regret at Most 4ḡp̄γ̄√(Td log((2d+T)/(2d))) + p̄D̄ for Arbitrary CovariatesResearch Paper

A retailer that sets prices online sees, before each sale, information about the customer, the product or the market: a covariate vector. Demand depends on the price and on these covariates through parameters the seller does not know, so the seller must learn the demand model while pricing. This dynamic pricing with covariates problem is a standard model in…

11 thms4 active usersReviewed
Operations ResearchProbabilityReinforcement Learning+1·Captain: iris

Regret Analysis of a Markov Policy Gradient Algorithm for Multiarm Bandits: SAMBA Converges to the Optimal Arm Almost Surely with Regret at Most NαΔlog⁡T+Q\frac{N}{\alpha\Delta}\log T + QαΔN​logT+QResearch Paper

Policy-gradient methods adjust the probabilities with which a learner picks actions by following an estimated gradient of its expected reward. They underlie much of modern reinforcement learning (Williams 1992; Sutton–Barto 2018), yet regret guarantees for them are scarce even in the simplest setting, the stochastic multi-armed bandit, where the benchmark…

24 thms4 active usersReviewed
Machine LearningStatistics·Captain: iris

Distributionally Robust Batch Contextual Bandits: The Learned KL-Robust Policy Has Robust Regret O((κ(n)(Π)+log⁡(2/ε))/(bηn))O((\kappa^{(n)}(\Pi)+\sqrt{\log(2/\varepsilon)})/(b\eta\sqrt{n}))O((κ(n)(Π)+log(2/ε)​)/(bηn​)) with Probability 1−ε1-\varepsilon1−εResearch Paper

In offline policy learning a decision maker holds logged data from a contextual bandit — contexts, the actions a logging policy took, and the rewards those actions earned — and must choose a policy from a class Π\PiΠ without further experimentation. Personalized medicine, online advertising and public-policy targeting all fit this template. The standard…

24 thms4 active usersReviewed
Operations ResearchProbability·Captain: Shuze Chen

Matching While Learning: Asymptotically optimal regret for decentralized matchingResearch Paper

A service platform may know which kinds of jobs are available while still learning the abilities of new workers. An assignment then has two effects: it produces a reward and supplies information that can improve later assignments. Limited job supply couples these decisions across workers. Sending one worker to a useful learning task consumes capacity that…

16 thms4 active usersReviewed
🏆Completed
Operations ResearchOptimizationProbability+1·Captain: Shuze Chen

Multi-armed Bandit Allocation Indices X: Jobs and Targets, Location–Scale Invariance and Semi-Markov Sampling ProcessesTextbook

A bandit sampling process models a population from which values are drawn one at a time: the values are independent with a distribution f(⋅∣θ)f(\cdot \mid \theta)f(⋅∣θ) whose parameter θ\thetaθ is unknown, the decision maker holds a prior on θ\thetaθ and updates it by Bayes' theorem, and each draw earns a reward. When several such populations compete for the same sampling…

11 thms4 active usersReviewed
🏆Completed
Dynamic ProgrammingLinear OptimizationOperations Research+2·Captain: Shuze Chen

Multi-armed Bandit Allocation Indices IX: Suboptimality Bounds for Branching Bandits and Index Policies on Parallel MachinesTextbook

Gittins index policies are optimal for a single server choosing among alternative bandit processes, and the achievable region method of Bertsimas and Niño-Mora (Math. Oper. Res. 1996) explains why: when the performance vectors of a scheduling system satisfy generalized conservation laws, the region they fill is a polytope whose optimal vertex is an index…

11 thms4 active usersReviewed
ProbabilityStatistics·Captain: mikedeng1

Asymptotically Optimal Sequential Design for Rank Aggregation: The ε-Greedy Generalized-Likelihood-Ratio Ranking Policies Attain the Minimal Bayes Risk as the Comparison Cost Tends to ZeroResearch Paper

Rank aggregation turns many local comparisons into one global ordering. In a sequential setting, the experimenter chooses which two items to compare next, sees a noisy binary answer, and decides when enough evidence has been collected. More comparisons can improve the ranking, but each comparison has a cost. This tradeoff appears in preference elicitation…

15 thms4 active usersReviewed
🏆Completed
Operations ResearchProbabilityStatistics·Captain: mikedeng1

The (Surprising) Sample Optimality of Greedy Procedures for Large-Scale Ranking and Selection 3: The Explore-First Greedy Procedure Is Consistent with a Budget Linear in kResearch Paper

Ranking and selection (R&S) asks which of kkk simulated systems has the largest mean performance when every observation is costly. It is a basic tool of simulation optimization: choosing a staffing level, an inventory policy or a network design from a large catalogue of candidates that can only be evaluated by noisy simulation. In the fixed-budget formulation…

18 thms4 active usersReviewed
🏆Completed
Operations ResearchProbabilityStatistics·Captain: mikedeng1

The (Surprising) Sample Optimality of Greedy Procedures for Large-Scale Ranking and Selection 2: Under the Slippage Configuration with a Common Variance the Greedy PCS Lower Bound Is TightResearch Paper

Ranking and selection (R&S) asks how to spend a fixed budget of simulation runs on kkk competing system designs so that the design with the largest mean performance is identified with high probability. When kkk runs into the thousands or millions, the relevant question is how the total budget must grow with kkk. Li, Fan and Hong (arXiv:2303.02951v3, published in M…

16 thms4 active usersReviewed
🏆Completed
Operations ResearchProbabilityStatistics·Captain: mikedeng1

The (Surprising) Sample Optimality of Greedy Procedures for Large-Scale Ranking and Selection 1: On a Budget Linear in k the Greedy Procedure Keeps the PCS Bounded Away from ZeroResearch Paper

Ranking and selection (R&S) is the problem of identifying, by simulation, the alternative with the largest mean performance among kkk candidates, when each alternative can only be observed with noise. It is a basic tool of simulation optimization, with roots in Bechhofer's indifference-zone formulation (Bechhofer 1954). Cloud and parallel computing have made…

12 thms4 active usersReviewed
Machine LearningOperations ResearchProbability·Captain: mikedeng1

Thompson Sampling with Information Relaxation Penalties I: For Natural Exponential Families the Suboptimality Gaps of TS, IRS.FH and IRS.V-Zero Are at Most 2√L[1/√ν + √(2 log T)(K/√ν + 2√(KT))]Research Paper

Thompson sampling (TS) is the most widely used heuristic for Bayesian multi-armed bandits: at each step it samples a parameter from the current posterior and plays the arm that is best for the sample. Its Bayesian regret has been analysed extensively (Russo and Van Roy, 2014; Bubeck and Liu, 2013). Min, Maglaras and Moallemi (arXiv:1902.04251) place TS…

9 thms4 active usersReviewed
Operations ResearchProbabilityStochastic Systems·Captain: mikedeng1

Learning Unknown Service Rates in Queues 3: In Heavy Traffic Every α-Consistent Scheduling Policy Has Average Queue-Regret at Least D(μ)(K−1) log t / (4 log log t) Up to Time (K−1)D(μ)/(4ε)Research Paper

A switch network is a set of UUU queues served by K≥UK\ge UK≥U servers. In every time slot each server can serve at most one queue, so a scheduler must pick a matching of queues to servers. Such systems model input-queued packet switches, crowdsourcing platforms that route job types to worker types, and wireless links with unknown channel quality. When the success…

10 thms4 active usersReviewed
Operations ResearchProbabilityStochastic Systems·Captain: mikedeng1

Learning Unknown Service Rates in Queues 1: Every α-Consistent Scheduling Policy Has Average Queue-Regret at Least λ_min D(μ)(1−α)(K−1)/(8t) for Infinitely Many tResearch Paper

Many service systems must route work to servers whose quality is not known in advance: crowdsourcing platforms matching job types to worker types, wireless schedulers assigning users to channels, call centers routing request classes to agent pools. Learning the unknown service rates is a multi-armed bandit problem, but the cost of a bad decision is not a…

8 thms4 active usersReviewed
Operations ResearchProbability·Captain: mikedeng1

A Primal-Dual Learning Algorithm for Personalized Dynamic Pricing with an Inventory Constraint: The Primal-Dual Learning Algorithm Has Regret at Most C(log n)^(2+δ) n^(−1/2)Research Paper

A firm selling a fixed stock of a perishable product over a finite season must set prices without knowing how demand responds to them. Online retail and travel platforms often see a customer's segment before quoting a price, so they can price-discriminate across consumer types. Learning a separate demand curve for each type, however, seems to cost more…

21 thms4 active usersReviewed
🏆Completed
Machine LearningProbability·Captain: mikedeng1

A Modern Introduction to Online Learning VIII: Tsallis-INF Has Adversarial Regret 32L∞√((d − 1)T) and Stochastic Pseudo-Regret 256L∞² Σ_i (1 + ln T)/Δ_iTextbook

In online learning with bandit feedback, the learner chooses one action and sees only that action's loss. This is the information pattern of sequential trials in which testing one option does not reveal the outcomes of the others. The learner must keep its cumulative loss close to that of a good fixed action while learning which actions are good. A…

6 thms4 active usersReviewed
🏆Completed
Machine LearningOperations Research·Captain: mikedeng1

Regret in Online Combinatorial Optimization I: For Every Learning Rate the Exponentially Weighted Forecaster EXP2 Has Full-Information Regret at Least 0.01 d^{3/2}√n on Some Action SetResearch Paper

Online combinatorial optimization is a repeated game in which a learner selects, round after round, a structured object (a subset of fixed size, a path, a matching, a spanning tree) encoded as a 0/1 vector, and an adversary assigns a cost to every coordinate. It models routing, ranking, scheduling and many other sequential decision problems in operations…

7 thms4 active users
🏆Completed
Machine LearningProbability·Captain: mikedeng1

A Tutorial on Thompson Sampling II: Under Thompson Sampling, Per-Period Expected Regret Splits into a Pessimism Term and a Width Term for Any History-Determined U_tResearch Paper

An online decision maker repeatedly chooses an action, observes an outcome, and receives a reward. In a Bayesian model, an unknown parameter controls the outcomes, and the decision maker begins with a prior distribution for that parameter. Thompson sampling acts by drawing a parameter from its current posterior distribution and choosing an action that would…

7 thms4 active users
🏆Completed
Operations ResearchProbabilityStatistics·Captain: mikedeng1

Some Aspects of the Sequential Design of Experiments II: Staying on Heads and Switching on Tails Loses at Most 3 − 2√2 per Toss in the Long RunResearch Paper

In 1952 Herbert Robbins asked how a statistician should draw a sample sequentially from two populations when the goal is the largest possible expected total, not a test or an estimate after the fact (Robbins 1952). This is the problem now called the two-armed bandit, and Robbins' paper is the usual starting point of the multi-armed bandit literature that…

12 thms4 active usersReviewed
🏆Completed
Operations ResearchProbabilityStatistics·Captain: mikedeng1

Dynamic Pricing Without Knowing the Demand Function: Risk Bounds and Near-Optimal Algorithms I: The Nonparametric Learn-then-Price Policy Has Regret at Most C(log n)^{1/2}/n^{1/4}Research Paper

A seller with a fixed stock of a single product and a finite selling season has to set prices without knowing how demand responds to them. This is the standard situation in revenue management: airline seats, hotel rooms, fashion goods and event tickets. The classical theory of dynamic pricing (Gallego and van Ryzin, 1994) assumes that the demand function …

15 thms4 active usersReviewed
PreviousPage 3 of 9Next

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paperProve2Me Team
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me