Prove2Me
Navigate
DiscoverCollectionsFormalpediaGalaxyBlogsUsersMomentumMy Missions+
Prove2Me
Log in

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

Bandit Algorithms

215 missions · 122 completed

Missions

Open93Completed122All215
Operations ResearchOptimization·Captain: Shuze Chen

Online Learning and Optimization for Revenue Management Problems with Add-on Discounts: Regret bound for UCB-Add-OnResearch Paper

Retailers sometimes offer a discount on a game only after a customer buys a console. Such add-on discounts let the customer choose individual additional products while the retailer controls both the original prices and the conditional offers. A retailer with limited transaction history must learn purchase probabilities while choosing these prices. Offering…

9 thms2 active usersReviewed
Machine LearningOperations ResearchProbability·Captain: mikedeng1

Robust Dynamic Assortment Optimization in the Presence of Outlier Customers: Active Elimination Has TOTE Regret O(ε̄K²T log T + (K²√ε̄ + √K)√(NT log³T) + K²N log²T) Under ε-ContaminationResearch Paper

A retailer that learns customer preferences online, for example an e-commerce platform choosing which products to display, observes purchases that are not all generated by the population it is trying to learn. Some arrivals are bots, fraudulent accounts or atypical buyers, and their choices can be arbitrary or deliberately misleading. Standard learning…

10 thms2 active usersReviewed
🏆Completed
Operations ResearchProbability·Captain: mikedeng1

Learning to Rank an Assortment of Products: The Simple Learning Algorithm Hooks Half the Optimal Ranking's Customers up to ε with Probability 1 − δResearch Paper

Online retailers display hundreds of products in a list or a grid, and most visitors never scroll far. A customer who finds nothing appealing near the top leaves without engaging at all, so the order of the first few slots decides whether a visit turns into a click, and through it into a sale. Ferreira, Parthasarathy and Sekar (SSRN 3395992; Management…

9 thms2 active usersReviewed
🏆Completed
Algorithmic Game TheoryMachine LearningReinforcement Learning·Captain: mikedeng1

V-Learning—A Simple, Efficient, Decentralized Algorithm for Multiagent Reinforcement Learning: Independent V-Learning Finds a Coarse Correlated Equilibrium with Gap O((H²S/K)Ξ(A,K/S,ι) + √(H⁵Sι/K))Research Paper

In multi-agent reinforcement learning several agents act in a shared, changing environment, each pursuing its own reward. The standard model is the Markov game (stochastic game), going back to Shapley (1953). When agents learn from experience rather than from a known model, the natural target is an equilibrium of the game, and the natural question is how…

10 thms2 active usersReviewed
Machine LearningOperations Research·Captain: mikedeng1

Hedging the Drift: Learning to Optimize under Non-Stationarity: the Bandit-over-Bandit algorithm has dynamic regret Õ(d^{2/3}B_T^{1/3}T^{2/3} + d^{1/2}T^{3/4}) without knowing B_TResearch Paper

Many sequential decision problems in operations research are learned online while the environment changes: demand responds to prices differently from week to week, click-through rates drift, and the value of an action in a recommendation or auto-loan pricing system decays as conditions move. The stochastic linear bandit is the standard model for such…

10 thms2 active usersReviewed
EconomicsOperations ResearchProbability·Captain: mikedeng1

Optimal Search and Discovery: Always Taking the Action with the Largest Purchase, Search or Discovery Value Is an Optimal PolicyResearch Paper

In the standard model of costly sequential search, a consumer knows from the start which products exist and pays to learn how much each is worth. Weitzman (Econometrica 1979) solved this problem with a single number per product, the reservation value: inspect in decreasing order of reservation values, and stop as soon as the best product found is worth more…

9 thms2 active usersReviewed
Linear algebraMachine LearningProbability·Captain: mikedeng1

The Elliptical Potential Lemma for General Distributions with an Application to Linear Thompson Sampling: For Any Prior and Noise, E[Σ Aₜᵀ Γₜ Aₜ] ≤ 2 max(σ², 1) log det(I + T Γ₁)Research Paper

In a linear bandit problem, a learner chooses a vector and observes a noisy scalar response that carries information about an unknown parameter. Regret analyses often need to control the sum of the learner's uncertainty in the directions it chooses. The familiar elliptical potential lemma does this when uncertainty is represented by the inverse of a…

7 thms2 active usersReviewed
Dynamic ProgrammingOperations ResearchOptimization+1·Captain: Shuze Chen

Multi-armed Bandit Allocation Indices VIII: Index Theorems for Non-Preemptive Jobs under Precedence Constraints and Out-ForestsTextbook

Scheduling jobs on a single machine under precedence constraints ("job iii must be completed before job jjj may be started") is one of the oldest problems of operations research, and with random service times and discounted rewards it is a family of alternative bandit processes that is not simple: the set of jobs that may be served changes as jobs are…

7 thms2 active usersReviewed
Operations ResearchProbabilityStatistics·Captain: mikedeng1

Balancing Optimal Large Deviations in Ranking and Selection: The BOLD Allocation Learns the Large Deviations Optimality Conditions With Probability OneResearch Paper

Ranking and selection asks which of MMM simulated alternatives has the largest mean, when each simulation run returns one noisy sample and the simulation budget is limited. A large literature allocates the budget using the large-deviations analysis of Glynn and Juneja (2004): for a static allocation in which alternative xxx receives a fixed fraction αx\alpha_xαx​ of…

10 thms2 active usersReviewed
🏆Completed
Algorithmic Game TheoryMachine LearningOperations Research·Captain: mikedeng1

Optimal No-regret Learning in Repeated First-price Auctions 3: Even with Full Information and v_t ≡ 1, Every Bidding Policy Has Worst-Case Expected Regret at Least c√TResearch Paper

Most online advertising inventory is sold one impression at a time in automated auctions, and since about 2019 the major ad exchanges have moved from second-price to first-price rules: the highest bidder wins and pays its own bid. A bidder in a first-price auction must trade off winning probability against the price paid, and that trade-off depends on the…

12 thms2 active usersReviewed
🏆Completed
Machine LearningOperations Research·Captain: mikedeng1

Optimal No-regret Learning in Repeated First-price Auctions 5: With Adversarial Contexts and K = (α+1)M ≤ T^(1/3), Every Policy for Partially Ordered Contextual Bandits Has Regret ≥ √(KT)/(32e³)Research Paper

A bidder in repeated first-price auctions faces a learning problem with two kinds of side information. Their private value is a context that changes every round, and the outcome of one bid reveals something about other bids (graph feedback across actions) and about other values (cross learning across contexts). Han, Zhou and Weissman (arXiv:2003.09795v7)…

12 thms2 active usersReviewed
🏆Completed
Algorithmic Game TheoryMachine LearningOperations Research·Captain: mikedeng1

Optimal No-regret Learning in Repeated First-price Auctions 6: If the HOB Law Switches at an Unknown Value Threshold, Every Bidding Policy Has Regret Ω(T)Research Paper

In a first-price auction the highest bidder wins and pays her own bid. Display advertising moved to first-price auctions around 2019, and a bidder who participates in millions of them faces a learning problem: at each round ttt she sees a private value vt∈[0,1]v_t\in[0,1]vt​∈[0,1] for the item, bids bt∈[0,1]b_t\in[0,1]bt​∈[0,1], and wins when btb_tbt​ is at least the highest other bid (HOB) mtm_tmt​,…

9 thms2 active usersReviewed
Machine LearningOperations Research·Captain: mikedeng1

Optimal No-regret Learning in Repeated First-price Auctions 4: Arm Elimination with Graph Feedback and Cross Learning Has Regret Õ(T ∧ √(αMT) ∧ √(KT)) with High ProbabilityResearch Paper

In many sequential decision problems, playing one action reveals the outcome of other actions too. A bidder in a first-price auction who bids bbb and sees the winning price learns whether every bid at or above that price would have won. A recommender that shows one item learns something about related items. Two kinds of side information recur. Graph feedback a…

8 thms2 active usersReviewed
🏆Completed
Machine LearningOperations ResearchProbability·Captain: mikedeng1

Thompson Sampling with Information Relaxation Penalties III: As T → ∞ the Action Laws of IRS.FH and IRS.V-Zero Converge to That of Thompson SamplingResearch Paper

Thompson sampling (TS) is the standard randomized heuristic for the Bayesian multi-armed bandit: at each decision it samples a parameter vector from the current posterior and pulls the arm that would be best if the sample were true. It ignores the remaining horizon. Min, Maglaras and Moallemi (arXiv:1902.04251, Management Science, 2025) obtain horizon-aware…

9 thms2 active usersReviewed
Harmonic AnalysisMachine LearningOperations Research+1·Captain: mikedeng1

Learning and Optimization with Seasonal Patterns 1: Regret Upper Bound for the Two-Stage Period-Estimation and Confidence-Bound PolicyResearch Paper

Many demand, traffic and click-through series are seasonal: their mean repeats after a fixed number of periods (a week of days, a year of months), while the length of that cycle is not known in advance. A retailer choosing among KKK promotions, or a platform choosing among KKK recommendations, then faces a multi-armed bandit whose arms have time-varying but…

17 thms2 active usersReviewed
Machine LearningOperations ResearchStatistics·Captain: mikedeng1

Online Learning and Decision-Making under Generalized Linear Model with High-Dimensional Data: The G-MCP-Bandit Algorithm Has Expected Cumulative Regret at Most Rmax(T0 + |K|) + C log(T + 1)Research Paper

Personalized decisions — which drug to prescribe, which product to recommend, which price to quote — are made one user at a time, from user features observed on arrival, and the effect of each decision is learned only from the users it was applied to. This is the contextual (covariate) bandit. In electronic records and online platforms the feature vector is…

10 thms2 active usersReviewed
🏆Completed
Machine LearningOperations ResearchReinforcement Learning·Captain: mikedeng1

Corruption-Robust Exploration in Episodic Reinforcement Learning I: Regret of SuperVIsed.C Is Bounded by the Bonus Sums of Its Critically Tolerant and Robust Base LearnersResearch Paper

Reinforcement-learning algorithms with regret guarantees, such as UCBVI (Azar, Osband & Munos 2017), assume that every episode is generated by the same Markov decision process. In practice a fraction of episodes can be generated by something else: fraudulent users of a recommender, sensor faults, a strategic participant. Corruption-robust learning asks for…

14 thms2 active usersReviewed
Operations ResearchProbabilityStochastic Systems·Captain: mikedeng1

Learning Unknown Service Rates in Queues 2: Q-UCB Has Queue-Regret at Most 6K v_u(t) log² t / t + (24.004 + UK)/(6t²) in the Late StageResearch Paper

Many service systems must match jobs to servers whose quality is not known in advance: crowdsourcing platforms routing tasks to worker types, wireless schedulers assigning users to channels, and switches in data networks. The scheduler learns server quality only by trying servers, and every poor assignment lets work pile up. The natural performance measure…

12 thms2 active usersReviewed
Machine LearningProbabilityReinforcement Learning·Captain: mikedeng1

(More) Efficient Reinforcement Learning via Posterior Sampling: PSRL Has Bayesian Regret O(τS√(AT log(SAT))) in Episodic Finite MDPsResearch Paper

An agent that interacts repeatedly with an unknown Markov decision process must trade exploration against exploitation. The dominant design principle for provably efficient reinforcement learning has been optimism in the face of uncertainty: UCRL2 (Jaksch, Ortner & Auer, 2010) and REGAL (Bartlett & Tewari, 2009) build confidence sets around the unknown…

14 thms2 active usersReviewed
Machine LearningProbabilityStatistics·Captain: mikedeng1

Bypassing the Monster: A Faster and Simpler Optimal Algorithm for Contextual Bandits Under Realizability: FALCON+ Has Regret ≤ 14.3·Σₘ√(K𝓔(τₘ₋₁−τₘ₋₂))(τₘ−τₘ₋₁) + τ₁ + √(8T log(2/δ)) w.p. 1−δResearch Paper

A contextual bandit is a sequential decision problem: in each round a learner sees a context (a user's features, a patient's covariates), picks one of KKK actions, and observes the reward of that action only. Applications include recommendation, dynamic pricing and adaptive clinical decisions. The difficulty is statistical and computational at once: the…

14 thms2 active usersReviewed
Machine LearningOperations ResearchStatistics·Captain: mikedeng1

Personalized Dynamic Pricing with Machine Learning: High-Dimensional Features and Heterogeneous Elasticity 2: Under Unknown Sparsity, Lasso-Based ILQX Pricing Has Regret at Most Cs√T(log d + log T)Research Paper

Online sellers of loans, insurance, travel and retail goods observe data about each arriving customer before quoting a price. Using that data to price individually is attractive only if the seller can learn, from its own sales, how demand responds to price for customers with given characteristics. Two features make this hard in practice. The number of…

6 thms2 active usersReviewed
Operations ResearchProbabilityStatistics·Captain: mikedeng1

Data-Based Dynamic Pricing and Inventory Control with Censored Demand and Limited Price Changes 1: In the Well-Separated Case, Algorithm-I with m Price Changes Has Regret at Most KT^(1/(m+1))Research Paper

Retailers set prices and stock levels without knowing how demand responds to price. They must learn the demand curve from their own sales while still earning money, which is the exploration–exploitation trade-off of dynamic pricing with learning (den Boer 2015). Two features of practice make the problem harder than the textbook bandit. First, demand is…

10 thms2 active usersReviewed
Machine LearningOperations ResearchStatistics·Captain: mikedeng1

Personalized Dynamic Pricing with Machine Learning: High-Dimensional Features and Heterogeneous Elasticity 1: Under Known Sparsity, ILSX Pricing Has Regret at Most Cs√T log TResearch Paper

A seller who changes prices while learning demand faces two costs: a price experiment may earn less revenue now, while insufficient experimentation leaves later prices poorly calibrated. Customer features make the balance harder because both willingness to buy and sensitivity to price may vary between customers. Ban and Keskin study this problem when the…

6 thms2 active usersReviewed
Machine LearningProbabilityStatistics·Captain: mikedeng1

The Multi-Armed Bandit Problem with Covariates I: Successive Elimination Has Expected Regret at Most 392γ²(1 + n/T)(K/Δ)log(TΔ²/18γ²) + nΔ⁻ Under a Random HorizonResearch Paper

In a multi-armed bandit, a decision maker repeatedly chooses an arm, observes a reward, and tries to learn which arm has the highest average reward while losing little reward during learning. A policy that requires the number of future pulls in advance cannot be used unchanged when stopping time is uncertain. Perchet and Rigollet study successive elimination…

10 thms2 active usersReviewed
Machine Learning·Captain: mikedeng1

An Optimal Algorithm for Stochastic and Adversarial Bandits III: α-Tsallis-INF With Gap-Tuned Asymmetric Regularization Has Logarithmic Pseudo-Regret When the Best Arm Is UniqueResearch Paper

In the multi-armed bandit problem a learner repeatedly chooses one of KKK actions and observes only the loss of the action it chose. Two models of the environment dominate the literature. In the stochastic model the losses of each arm are drawn i.i.d. from a fixed distribution, and the optimal pseudo-regret grows like ∑i≠i∗log⁡(T)/Δi\sum_{i\ne i^*}\log(T)/\Delta_i∑i=i∗​log(T)/Δi​, where …

14 thms2 active usersReviewed
PreviousPage 7 of 9Next

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paperProve2Me Team
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me