Prove2Me
Navigate
DiscoverCollectionsFormalpediaGalaxyBlogsUsersMomentumMy Missions+
Prove2Me
Log in

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

Bandit Algorithms

215 missions · 122 completed

Missions

Open93Completed122All215
Machine LearningReinforcement LearningStatistics·Captain: iris

Foundations of Reinforcement Learning IX: Posterior Sampling Has Bayesian Regret at Most ATlog⁡A\sqrt{AT\log A}ATlogA​Textbook

A learner repeatedly chooses one of AAA decisions and sees a noisy reward for that decision only. In the Bayesian version of this multi-armed bandit problem, the unknown reward distributions are drawn from a known prior. Posterior sampling, also called Thompson sampling or probability matching, plays each decision with the probability that it is optimal given…

20 thms6 active usersReviewed
Graph TheoryMachine Learning·Captain: iris

The Price of Partial Information with Switching Costs: A Complete Characterization for Feedback GraphsResearch Paper

Online learning algorithms randomize to hedge against an adversary, and independent resampling makes them change action often. When changing a route, a deployed model or a treatment has a cost of its own, the right benchmark charges one unit per change. With full information this charge leaves the minimax regret at order Tlog⁡K\sqrt{T\log K}TlogK​ (Kalai and Vempala,…

40 thms5 active usersReviewed
Machine LearningOperations ResearchProbability·Captain: iris

Approximately Stationary Bandits with Knapsacks: Algorithm 1 Earns ρ+σr(σc−ρ)+\rho + \sigma_r(\sigma_c - \rho)^+ρ+σr​(σc​−ρ)+ of OPTFD\mathrm{OPT}_{FD}OPTFD​ Against Adaptive (σr,σc)(\sigma_r, \sigma_c)(σr​,σc​)-Stationary AdversariesResearch Paper

In Bandits with Knapsacks (BwK) a player repeatedly picks one of KKK actions; each choice yields a reward and consumes ddd limited resources, and play stops as soon as a budget runs out. The model covers dynamic pricing with limited inventory, ad auctions with a spending cap and crowdsourcing with a payment budget (Badanidiyuru, Kleinberg, Slivkins 2018). Its…

17 thms5 active usersReviewed
Algorithmic Game TheoryConvex Optimization·Captain: iris

Doubly Optimal No-Regret Online Learning in Strongly Monotone Games with Bandit Feedback: Barrier Bandit Mirror Descent Reaches the Nash Equilibrium at Last-Iterate Rate O~(nT−1/2)\tilde O(nT^{-1/2})O~(nT−1/2)Research Paper

In many operations problems several decision makers act repeatedly on a shared system and see only the payoff they receive: firms setting quantities in a Cournot market observe the market price, sellers in a pricing game observe their revenue, bidders in a Kelly auction observe their allocation. None of them knows the reward functions or their gradients. Ban…

18 thms5 active usersReviewed
Machine LearningReinforcement Learning·Captain: iris

Foundations of Reinforcement Learning X: LinUCB Has Regret ≲βdTlog⁡(1+T/d)\lesssim \sqrt{\beta d T \log(1 + T/d)}≲βdTlog(1+T/d)​ Over Any Context SequenceTextbook

Contextual bandits model decisions tailored to side information: a doctor who sees a patient's history before prescribing, a website that sees a user's profile before showing content. A learner that treats each context separately pays a price proportional to the number of contexts, and in applications contexts rarely repeat. Linear contextual bandits escape…

22 thms5 active usersReviewed
Operations ResearchOptimizationProbability·Captain: Shuze Chen

Multi-armed Bandit Allocation Indices VII: Freezing Rules, Forwards Induction and the Near-Optimality of ε-Index* PoliciesTextbook

A multi-armed bandit in the sense of Gittins is a sequential allocation problem: several independent Markov reward processes compete for a single resource, at each decision time exactly one of them is operated, and the goal is to maximize the expected total discounted reward. The Gittins index theorem says that it is optimal to operate, at every decision…

11 thms5 active usersReviewed
Operations ResearchStatistics·Captain: mikedeng1

Dynamic Assortment Personalization in High Dimensions 1: The Nuclear-Norm Explore-Then-Exploit Policy Has Regret at Most ((Cr(m+n)+3) log T + 1)ωResearch Paper

An online retailer that serves many kinds of customers must decide, for each arriving customer, which handful of products to display. Showing too many items is impossible (screen space, shelf space), and which items sell depends on who is looking. The retailer does not know the customers' preferences in advance and learns them only from the choices…

22 thms5 active usersReviewed
Dynamic ProgrammingOperations ResearchProbability·Captain: mikedeng1

Bandit Processes and Dynamic Allocation Indices: A Policy Is Optimal If and Only If It Almost Surely Continues a Bandit Process of Maximal Dynamic Allocation IndexResearch Paper

The multi-armed bandit problem asks how to share effort sequentially among several independent projects, each of which evolves only while it is worked on. It models the scheduling of jobs on a single machine, the order in which to explore candidate sites, sequential clinical trials and the allocation of research effort among competing projects. Before 1972…

8 thms5 active usersReviewed
Machine LearningProbabilityReinforcement Learning·Captain: iris

Introducing Feedback Delays to Non-Delayed Adversarial MDPs: Optimal √S and √K Dependencies in Tabular and Linear AMDPsResearch Paper

An episodic adversarial Markov decision process (AMDP) models a learner who repeats an HHH-step task for KKK episodes while an adversary fixes the loss function of every episode in advance. The learner sees only the losses of the state-action pairs it visits (bandit feedback), and it does not know the transition law. The standard measure of performance is the re…

82 thms4 active usersReviewed
Machine LearningOperations ResearchProbability·Captain: iris

No-Regret Revenue Maximization in Sequential Market Making Under Inventory ConstraintsResearch Paper

A market maker quotes two prices for an asset: a bid, at which it buys, and an ask, at which it sells. It earns the spread when both sides trade, and it carries inventory risk when they do not balance: a maker that buys much more than it sells is exposed to the price of the asset itself, which defeats the purpose of quoting. Classical market-making models…

29 thms4 active usersReviewed
Machine LearningOperations ResearchStatistics·Captain: iris

Dynamic Batch Learning in High-Dimensional Sparse Linear Contextual Bandits: LASSO Batch Greedy Learning Has Regret at Most O~(M3/2Ts0(T/s0)1/(2(2M−1))/(γρ))\tilde O(M^{3/2}\sqrt{Ts_0}(T/s_0)^{1/(2(2^M-1))}/(\gamma\rho))O~(M3/2Ts0​​(T/s0​)1/(2(2M−1))/(γρ))Research Paper

Personalized medicine, online advertising and news recommendation all choose among a few actions using a very large number of features per decision, of which only a few matter. They also rarely update a policy after every decision: clinical trials report in stages, and production recommenders are retrained in batches. Dynamic batch learning models this…

44 thms4 active usersReviewed
Machine LearningOperations ResearchProbability·Captain: iris

Technical Note—On Dynamic Pricing with Covariates: UCB Pricing Has Regret at Most 4ḡp̄γ̄√(Td log((2d+T)/(2d))) + p̄D̄ for Arbitrary CovariatesResearch Paper

A retailer that sets prices online sees, before each sale, information about the customer, the product or the market: a covariate vector. Demand depends on the price and on these covariates through parameters the seller does not know, so the seller must learn the demand model while pricing. This dynamic pricing with covariates problem is a standard model in…

11 thms4 active usersReviewed
Operations ResearchProbabilityReinforcement Learning+1·Captain: iris

Regret Analysis of a Markov Policy Gradient Algorithm for Multiarm Bandits: SAMBA Converges to the Optimal Arm Almost Surely with Regret at Most NαΔlog⁡T+Q\frac{N}{\alpha\Delta}\log T + QαΔN​logT+QResearch Paper

Policy-gradient methods adjust the probabilities with which a learner picks actions by following an estimated gradient of its expected reward. They underlie much of modern reinforcement learning (Williams 1992; Sutton–Barto 2018), yet regret guarantees for them are scarce even in the simplest setting, the stochastic multi-armed bandit, where the benchmark…

24 thms4 active usersReviewed
Machine LearningStatistics·Captain: iris

Distributionally Robust Batch Contextual Bandits: The Learned KL-Robust Policy Has Robust Regret O((κ(n)(Π)+log⁡(2/ε))/(bηn))O((\kappa^{(n)}(\Pi)+\sqrt{\log(2/\varepsilon)})/(b\eta\sqrt{n}))O((κ(n)(Π)+log(2/ε)​)/(bηn​)) with Probability 1−ε1-\varepsilon1−εResearch Paper

In offline policy learning a decision maker holds logged data from a contextual bandit — contexts, the actions a logging policy took, and the rewards those actions earned — and must choose a policy from a class Π\PiΠ without further experimentation. Personalized medicine, online advertising and public-policy targeting all fit this template. The standard…

24 thms4 active usersReviewed
Operations ResearchProbability·Captain: Shuze Chen

Matching While Learning: Asymptotically optimal regret for decentralized matchingResearch Paper

A service platform may know which kinds of jobs are available while still learning the abilities of new workers. An assignment then has two effects: it produces a reward and supplies information that can improve later assignments. Limited job supply couples these decisions across workers. Sending one worker to a useful learning task consumes capacity that…

16 thms4 active usersReviewed
ProbabilityStatistics·Captain: mikedeng1

Asymptotically Optimal Sequential Design for Rank Aggregation: The ε-Greedy Generalized-Likelihood-Ratio Ranking Policies Attain the Minimal Bayes Risk as the Comparison Cost Tends to ZeroResearch Paper

Rank aggregation turns many local comparisons into one global ordering. In a sequential setting, the experimenter chooses which two items to compare next, sees a noisy binary answer, and decides when enough evidence has been collected. More comparisons can improve the ranking, but each comparison has a cost. This tradeoff appears in preference elicitation…

15 thms4 active usersReviewed
Machine LearningOperations ResearchProbability·Captain: mikedeng1

Thompson Sampling with Information Relaxation Penalties I: For Natural Exponential Families the Suboptimality Gaps of TS, IRS.FH and IRS.V-Zero Are at Most 2√L[1/√ν + √(2 log T)(K/√ν + 2√(KT))]Research Paper

Thompson sampling (TS) is the most widely used heuristic for Bayesian multi-armed bandits: at each step it samples a parameter from the current posterior and plays the arm that is best for the sample. Its Bayesian regret has been analysed extensively (Russo and Van Roy, 2014; Bubeck and Liu, 2013). Min, Maglaras and Moallemi (arXiv:1902.04251) place TS…

9 thms4 active usersReviewed
Operations ResearchProbabilityStochastic Systems·Captain: mikedeng1

Learning Unknown Service Rates in Queues 3: In Heavy Traffic Every α-Consistent Scheduling Policy Has Average Queue-Regret at Least D(μ)(K−1) log t / (4 log log t) Up to Time (K−1)D(μ)/(4ε)Research Paper

A switch network is a set of UUU queues served by K≥UK\ge UK≥U servers. In every time slot each server can serve at most one queue, so a scheduler must pick a matching of queues to servers. Such systems model input-queued packet switches, crowdsourcing platforms that route job types to worker types, and wireless links with unknown channel quality. When the success…

10 thms4 active usersReviewed
Operations ResearchProbabilityStochastic Systems·Captain: mikedeng1

Learning Unknown Service Rates in Queues 1: Every α-Consistent Scheduling Policy Has Average Queue-Regret at Least λ_min D(μ)(1−α)(K−1)/(8t) for Infinitely Many tResearch Paper

Many service systems must route work to servers whose quality is not known in advance: crowdsourcing platforms matching job types to worker types, wireless schedulers assigning users to channels, call centers routing request classes to agent pools. Learning the unknown service rates is a multi-armed bandit problem, but the cost of a bad decision is not a…

8 thms4 active usersReviewed
Operations ResearchProbability·Captain: mikedeng1

A Primal-Dual Learning Algorithm for Personalized Dynamic Pricing with an Inventory Constraint: The Primal-Dual Learning Algorithm Has Regret at Most C(log n)^(2+δ) n^(−1/2)Research Paper

A firm selling a fixed stock of a perishable product over a finite season must set prices without knowing how demand responds to them. Online retail and travel platforms often see a customer's segment before quoting a price, so they can price-discriminate across consumer types. Learning a separate demand curve for each type, however, seems to cost more…

21 thms4 active usersReviewed
Machine LearningOperations ResearchProbability·Captain: mikedeng1

Stochastic Linear Optimization under Bandit Feedback 1: The Regret Bound of ConfidenceBallResearch Paper

In stochastic linear optimization under bandit feedback a learner repeatedly chooses a decision xtx_txt​ from a fixed set D⊆RnD\subseteq\mathbb R^nD⊆Rn and observes only the noisy cost of that one decision, whose expectation is a fixed but unknown linear function μ⊤xt\mu^\top x_tμ⊤xt​. The model covers online routing, ad and product selection with feature vectors, and any…

24 thms4 active users
Graph TheoryMachine Learning·Captain: iris

Near-Optimal First-Order Regret for Bandits with Feedback GraphsResearch Paper

In adversarial online learning with a feedback graph, a learner repeatedly picks one of KKK arms and, besides its own loss, observes the losses of the arms its choice points to in a directed graph. The model interpolates between prediction with expert advice (every arm reveals every loss) and multi-armed bandits (each arm reveals only itself) (Mannor–Shamir…

83 thms3 active usersReviewed
Machine LearningProbability·Captain: iris

An Efficient Minimax-Optimal Algorithm for Adversarial m-Set BanditsResearch Paper

In an adversarial mmm-set bandit problem a learner repeatedly picks mmm of ddd items, an adversary assigns losses to the items, and the learner observes only the total loss of the items it picked. The model covers ranking a fixed number of results, choosing a fixed-size portfolio, or scheduling mmm out of ddd resources when only aggregate feedback is available. It is…

41 thms3 active usersReviewed
Machine Learning·Captain: iris

Follow the Perturbed Leader for Linear Bandits: T\sqrt{T}T​ Regret over General Action SetsResearch Paper

In an adversarial linear bandit a learner repeatedly picks an action ata_tat​ from a set A⊂RdA\subset\mathbb R^dA⊂Rd, an adversary picks a reward function, and the learner sees only the reward of the action it played. The problem models online routing, ranking and combinatorial decision making with partial feedback, and its minimax regret over general action sets is …

47 thms3 active usersReviewed
ProbabilityStatistics·Captain: iris

Multi-armed Bandit Experimental Design: Online Decision-making and Adaptive Inference: EXP3EG Estimates Every Treatment Effect Within 4(2+2K2(1+e2))log⁡(2/δ)/t1−α4(2+2K^2(1+e^2))\log(2/δ)/\sqrt{t^{1-α}}4(2+2K2(1+e2))log(2/δ)/t1−α​ with Probability 1−δ1-δ1−δResearch Paper

An online experiment has two audiences. The experimenter who cares about welfare wants to stop assigning people to inferior treatments as quickly as possible; the statistician wants an accurate estimate of how much better one treatment is than another. A multi-armed bandit policy serves the first and starves the second: an arm that is played rarely is…

12 thms3 active usersReviewed
PreviousPage 1 of 4Next

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paperProve2Me Team
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me