Summability of discounted Markov-bandit round rewards
ProvedBanditAlgorithm.summable_discounted_markovBanditRoundRewardbanditsgittins-indexprobability
Under the discounted absolute-reward integrability assumption and , the sequence of expected discounted rewards generated by any finite-armed Markov bandit policy is summable:
converges absolutely, and hence converges as a real series.
This theorem is the convergence interface needed to pass finite-horizon reward inequalities to infinite-horizon discounted bandit values.
Preamble
import Definitions.Def_GittinsIndex
Formal statement
open MeasureTheory ProbabilityTheory ENNReal
theorem BanditAlgorithm.summable_discounted_markovBanditRoundReward
{k : ℕ} {S : Type*} [MeasurableSpace S]
(P : Kernel S S) [IsMarkovKernel P]
{r : S → ℝ} (hr : Measurable r)
{α : ℝ} (hα0 : 0 < α) (hα1 : α < 1)
(hint : DiscountedRewardIntegrable P r α)
(π : MarkovBanditPolicy k S) (x : Fin k → S) :
Summable (fun n ↦ α ^ n * markovBanditRoundReward P r π x n) := by
sorry
Source
Tor Lattimore and Csaba Szepesvári, Bandit Algorithms, Assumption 35.6 and §35.4, printed pp. 447 and 450–453 (free PDF pp. 455 and 458–461), https://tor-lattimore.com/downloads/book/book.pdf