Prove2Me
Navigate
DiscoverCollectionsFormalpediaGalaxyBlogsUsersMomentumMy Missions+
Prove2Me
Log in

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

Bandit Algorithms

215 missions · 122 completed

Missions

Open93Completed122All215
Machine LearningOperations ResearchProbability·Captain: iris

Maximal Objectives in the Multi-armed Bandit with Applications: ADA-ETC Has an Explicit Instance-Dependent Regret Bound and Worst-Case Regret O(K^{1/3}T^{2/3}√log K) for the Max ObjectiveResearch Paper

In the classical stochastic multi-armed bandit, a decision-maker pulls one of KKK arms at each of TTT times and is judged by the sum of all rewards collected. Ozbay and Kamble (arXiv:2006.06853v6, later in Management Science, 2024) study a different objective: the decision-maker is judged by the largest cumulative reward of a single arm at the end of the…

11 thms3 active usersReviewed
Dynamic ProgrammingMarkov ChainOperations Research·Captain: iris

Unichain and Aperiodicity Are Sufficient for Asymptotic Optimality of Average-Reward Restless Bandits: The Set-Expansion Policy Has Optimality Gap at Most 256 r_max λ_W²|S|²/(β²√N)Research Paper

A restless bandit couples NNN identical Markov decision processes through a budget: at every time step exactly αN\alpha NαN of the NNN arms must be pulled, and every arm changes state whether it is pulled or not. The model, introduced by Whittle (1988), describes scheduling of machines or patients, sensor management, queueing control and recommendation. Computing…

23 thms3 active usersReviewed
Machine LearningOperations ResearchProbability·Captain: iris

Delay-Adaptive Learning in Generalized Linear Contextual Bandits: Under Independent Delays with a Tail Envelope, DUCB-GLCB Satisfies the Regret Bound (16) with Probability at Least 1 − 5δResearch Paper

In many sequential decision problems the outcome of a decision arrives late. A recommendation's conversion, a clinical response or a sale is observed hours or weeks after the action, and in the meantime the decision maker has to keep acting. Contextual bandits model the decision side: at each round a learner sees side information, chooses one of KKK actions…

18 thms3 active usersReviewed
Machine LearningOperations ResearchStatistics·Captain: iris

Rate-Optimal Bayesian Simple Regret in Best Arm Identification: Two-Stage Exploration Has Bayesian Simple Regret at Most Copt/T′+o(1/T)C_{\mathrm{opt}}/T' + o(1/T)Copt​/T′+o(1/T)Research Paper

A clinical trial, an A/B test or a simulation study often ends with one decision: which treatment to adopt. Multi-armed bandit theory calls this best arm identification. When the arm means are fixed, the frequentist simple regret after TTT pulls decays exponentially at each instance, with a rate that degenerates as the gap closes. A practitioner who has a…

39 thms3 active usersReviewed
Machine LearningProbability·Captain: iris

A General Framework for Bandit Problems Beyond Cumulative Objectives: U-UCB has regret O(log T / T) for every stable, smooth EDPM with a linear gapResearch Paper

In the classical stochastic multi-armed bandit a decision maker is scored by the sum of the rewards it collects. Many applications score the collected rewards differently: a portfolio manager cares about the conditional value at risk (CVaR) or the Sharpe ratio of the returns, a clinical trial about a quantile, a risk-averse controller about mean–variance.…

16 thms3 active usersReviewed
Machine LearningReinforcement LearningStatistics·Captain: iris

Foundations of Reinforcement Learning XII: E2D Regret Is at Most the Decision-Estimation Coefficient Times T Plus γ·EstSqTextbook

In a structured bandit a learner repeatedly picks a decision πt\pi^tπt from a possibly huge or continuous set Π\PiΠ and observes a noisy reward with mean f⋆(πt)f^\star(\pi^t)f⋆(πt), where f⋆f^\starf⋆ belongs to a known class F\mathcal FF of functions Π→R\Pi\to\mathbb RΠ→R. The optimal regret depends on F\mathcal FF in ways that are hard to predict: optimism (UCB) and posterior sampling…

27 thms3 active usersReviewed
Operations Research·Captain: Shuze Chen

Joint Learning and Optimization for Multi-Product Pricing (and Ranking) Under a General Cascade Click Model: The regret bound for UCB-CascadeResearch Paper

A seller displaying several products must learn more than the probability of a sale. A low purchase rate can reflect customers leaving before reaching a product, declining to click, or clicking and then deciding against purchase. These mechanisms matter because changing an early product's price changes both its sales and the number of customers who reach…

11 thms3 active usersReviewed
Operations ResearchProbability·Captain: Shuze Chen

Multimodal Dynamic Pricing: High-probability regret for smooth multimodal demandResearch Paper

A retailer learning demand must choose prices before knowing which price earns the most. Each sale provides information only at the price actually posted. A demand curve can have several locally attractive regions, so a method that searches near one apparent optimum can overlook a better region. Wang, Chen and Simchi-Levi study pricing under smoothness…

11 thms3 active usersReviewed
Algorithmic Game TheoryOperations ResearchProbability·Captain: mikedeng1

Game of Thrones: Fully Distributed Learning for Multiplayer Bandits: Without Communication, the Game of Thrones Algorithm Has Expected Total Regret O(log^{1+δ} T)Research Paper

In a multiplayer multi-armed bandit, several agents repeatedly choose among the same finite set of arms, and two agents that choose the same arm collide and receive nothing. The model comes from cognitive radio and wireless networks: users share channels, a channel used by two users at once carries neither transmission, and the quality of a channel differs…

13 thms3 active usersReviewed
Machine LearningProbability·Captain: mikedeng1

Online Learning over a Finite Action Set with Limited Switching: An Algorithm for Prediction From Experts Whose Regret and Number of Switches Are Both O(√(T log n log(1/δ))) with Probability 1 − δResearch Paper

In prediction from experts (PFE), an online learner chooses one of nnn actions in each of TTT rounds, and an adversary assigns each action a loss in [0,1][0,1][0,1]. Performance is measured by regret, the learner's total loss minus that of the best single action in hindsight. In many applications changing the action is itself costly: re-routing traffic, re-balancing a…

10 thms3 active usersReviewed
Operations ResearchProbabilityStatistics·Captain: mikedeng1

Bayesian Sequential Learning for Clinical Trials of Multiple Correlated Medical Interventions: The cPDE Allocation Policy Is Asymptotically ConsistentResearch Paper

A clinical trial that compares several treatments for the same condition must decide, patient by patient, which treatment the next patient receives. When the treatments are related (doses of one drug, drugs of one class), an outcome observed on one arm is informative about the others, and a good allocation rule should exploit that correlation. Chick, Gans…

20 thms3 active users
Machine LearningReinforcement Learning·Captain: mikedeng1

The Statistical Complexity of Interactive Decision Making IX: Posterior Sampling Bounds the Square-Loss DEC by 2Δ + 24·θ·log²(γ)/γ, θ the Disagreement CoefficientResearch Paper

In a structured bandit problem a learner repeatedly chooses a decision π\piπ from a set Π\PiΠ and observes a noisy reward whose mean fM(π)f^M(\pi)fM(π) is an unknown function from a known class FM\mathcal{F}_{\mathcal{M}}FM​. Classical analyses of such problems measure the complexity of FM\mathcal{F}_{\mathcal{M}}FM​ by the eluder dimension (Russo and Van Roy, 2013; 2014) or the st…

13 thms3 active users
Machine LearningReinforcement Learning·Captain: mikedeng1

The Statistical Complexity of Interactive Decision Making VIII: Linear Bandits on the Unit Ball Have Localized DEC at Least d/(12γ)Research Paper

In a linear bandit a learner repeatedly chooses a decision π\piπ from a set Π⊆Rd\Pi \subseteq \mathbb{R}^dΠ⊆Rd and observes a noisy reward whose mean is ⟨θ⋆,π⟩\langle \theta^\star, \pi\rangle⟨θ⋆,π⟩ for an unknown parameter θ⋆\theta^\starθ⋆. It is the standard model for sequential decisions with many correlated actions, such as ad placement, routing and recommendation, and its…

11 thms3 active usersReviewed
Machine LearningProbabilityStatistics·Captain: mikedeng1

Thompson Sampling: An Asymptotically Optimal Finite-Time Analysis: Bernoulli Thompson Sampling Has Regret (1+ε)Σ Δ_a(ln T + ln ln T)/K(μ_a, μ*) + CResearch Paper

Thompson Sampling is the oldest bandit algorithm: Thompson proposed it in 1933 for allocating patients between two treatments (Thompson 1933). It keeps a posterior distribution over the unknown mean reward of every action, draws one sample from each posterior, and plays the action whose sample is largest. It is simple, widely used, and in experiments…

17 thms3 active usersReviewed
Operations ResearchProbabilityStatistics·Captain: mikedeng1

Optimal Dynamic Assortment Planning with Demand Learning: The Separation-Based Policy Has Regret at Most κ₁⌈N/C⌉ log T + K̄₁Research Paper

A retailer who can display only a limited number of products must decide which assortment to show each arriving customer, without knowing how attractive each product is. Offering a product reveals how often it is bought, but offering a poor product costs revenue. This exploration–exploitation trade-off arises in fast fashion, where the selling season is…

13 thms3 active usersReviewed
Complexity TheoryQuantum Information·Captain: iris

Gap-Agnostic Ground-State Preparation via Quantum BanditsResearch Paper

Preparing the ground state (the lowest-energy eigenvector) of a Hamiltonian HHH is a basic task in quantum simulation, quantum chemistry and condensed-matter physics. In full generality it is hard: deciding the ground energy of local Hamiltonians to inverse-polynomial precision is QMA-complete (Kitaev–Shen–Vyalyi 2002). Efficient algorithms therefore assume…

52 thms2 active usersReviewed
ProbabilityStatistics·Captain: iris

Minimum-Mean Testing under Metric Constraints:Shared Information, Certificates, and AdaptivityResearch Paper

Many verification tasks ask whether every one of kkk noisy quantities clears a threshold: a batch passes inspection only if every component meets its specification, and an update is deployed only if every task benefits. When the quantities are known to be related, one observation carries evidence about several possible failures at once. Chaomeng Chen and…

85 thms2 active usersReviewed
Machine Learning·Captain: iris

Online Learning with Ordered Experts (via Two-point Feedback)Research Paper

Many online decisions choose a point on an ordered scale: a posted price, a threshold, an inventory level. When a class of experts recommends such points, the learner would like to compete with the best expert while paying for only a few loss evaluations per round. Zuo (Online Learning with Ordered Experts (via Two-point Feedback), STOC 2027 submission)…

40 thms2 active usersReviewed
Machine LearningOperations Research·Captain: iris

Decoupling Corruption and Horizon in Robust Contextual PricingResearch Paper

In dynamic pricing a seller repeatedly posts a price to an arriving buyer and learns only whether the buyer bought. In the contextual version, each buyer comes with an observable feature vector, and the buyer's valuation is a fixed but unknown linear function of it. The seller must learn that linear function from yes/no answers while losing as little…

15 thms2 active usersReviewed
Machine LearningOperations Research·Captain: iris

Square-Root Regret for Adversarial Multiplayer Bandits without Collision Information or Shared RandomnessResearch Paper

Consider mmm devices sharing KKK wireless channels. In each time slot every device transmits on one channel, two devices on the same channel both fail, channel qualities change arbitrarily, and a device learns only what it transmitted itself. A device that transmitted nothing cannot tell a bad channel from a collision. This is the multiplayer bandit problem…

43 thms2 active usersReviewed
Operations ResearchStochastic Systems·Captain: iris

Efficient Decentralized Multi-agent Learning in Asymmetric Bipartite Queueing Systems: Under DAM.FE the Time-Averaged Weighted Queue Length Is O(K²(log N + K)L_check/ε³ + K·min(T,T₀)²/(εT))Research Paper

Many service systems match a set of agents with work to a set of servers in every time slot: devices sharing radio channels in cognitive radio, workers and tasks on an online service platform, customers and providers in a marketplace. When the matching is computed centrally and the service rates are known, the MaxWeight policy, which in every slot serves a…

22 thms2 active usersReviewed
Operations ResearchProbabilityRevenue Management and Pricing·Captain: iris

Continuous Assortment Optimization with Logit Choice Probabilities under Incomplete Information: The Stochastic Approximation Policy SAP(α, β) Has Worst-Case Regret O(log T)Research Paper

A retailer choosing which products to display, a platform choosing which items to recommend, and a broadcaster choosing which content to stream all face an assortment problem: offer a set of products, observe what customers buy, and earn the revenue of the purchased item. The standard demand model is the multinomial logit (MNL) model, under which the…

17 thms2 active usersReviewed
Operations ResearchProbabilityStatistics·Captain: iris

Weak Signal Asymptotics for Sequentially Randomized Experiments: Sample Paths of Lipschitz Sequentially Randomized Markov Experiments Converge Weakly to the Unique Diffusion Limit (3.1)Research Paper

Sequential experiments choose which treatment to try next from the outcomes seen so far. Multi-armed bandit algorithms such as Thompson sampling, tempered greedy rules and Luce's rule are the standard examples, and they are used in clinical trials, online platforms and adaptive A/B testing. Their behaviour is usually studied either in the large-signal…

14 thms2 active usersReviewed
Machine LearningOperations ResearchStatistics·Captain: iris

A Simple and Optimal Policy Design with Safety Against Heavy-Tailed Risk for Stochastic Bandits: SE and UCB Have O(KTln⁡T)O(\sqrt{KT\ln T})O(KTlnT​) Regret and an e−Ω(x/KT)e^{-\Omega(x/\sqrt{KT})}e−Ω(x/KT​) Regret TailResearch Paper

Stochastic multi-armed bandit policies are judged almost always by their expected regret. In operations settings such as dynamic pricing, assortment testing or clinical allocation, a decision maker also cares about the probability of a bad outcome: a policy whose regret is small on average can still incur regret linear in the horizon TTT with non-negligible…

21 thms2 active usersReviewed
Machine LearningOperations Research·Captain: iris

Shrinking the Upper Confidence Bound: A Dynamic Product Selection Problem for Urban Warehouses: ConsUCB Has Regret at Most K + αK + 10α√(dKT log(KT)) with Probability at Least 1 − δResearch Paper

An online retailer choosing which low-demand products to display, and to stock in its urban warehouses, faces an unusual learning problem. Thousands of products must be chosen at once, the selection can change only a few times per season, and sales data arrive only for the products that were offered. Jin, Simchi-Levi, Wang, Wang and Yang (arXiv:1903.07844v2)…

19 thms2 active usersReviewed
PreviousPage 2 of 4Next

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paperProve2Me Team
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me