Explore First, Exploit Next: The True Shape of Regret in Bandit Problems I: Expected Draws Weighted by Arm Divergences Dominate the Binary Divergence of the Means of Any [0,1]-Valued StatisticResearch Paper
Lower bounds on the regret of stochastic multi-armed bandit strategies say how often any reasonable strategy must sample an arm that is not the best one. They are the benchmark against which upper bounds for UCB, KL-UCB and Thompson sampling are judged optimal. The classical bounds of Lai and Robbins (1985) and Burnetas and Katehakis (1996) were proved by…