Discounted stopping-block charge identity
ProvedBanditAlgorithm.discountedStoppedSum_sub_chargebanditsinfinite-seriesoptimal-stopping
For a trajectory whose discounted absolute rewards are summable, the net reward of a stopping block at constant charge is
The assumption guarantees summability of the stopped discounted time series.
Preamble
import Mathlib.Analysis.Normed.Group.InfiniteSum import Mathlib.Analysis.SpecificLimits.Normed import Definitions.Def_GittinsIndex open MeasureTheory ProbabilityTheory ENNReal
Formal statement
theorem BanditAlgorithm.discountedStoppedSum_sub_charge
{S : Type*} {α γ : ℝ} (hα0 : 0 ≤ α) (hα1 : α < 1)
(r : S → ℝ) (τ : (ℕ → S) → ℕ∞) (ω : ℕ → S)
(hsum : Summable (fun t : ℕ ↦ α ^ t * |r (ω t)|)) :
discountedStoppedSum α (fun y ↦ r y - γ) τ ω =
discountedStoppedSum α r τ ω -
γ * discountedStoppedSum α (fun _ : S ↦ 1) τ ω := by
sorrySource
Lattimore and Szepesvári, Bandit Algorithms (CUP 2020), Eq. (35.7) on printed p.448 and Part 1 of the proof of Theorem 35.9 on printed p.452. This is the pathwise algebraic accounting identity for each prevailing-charge block.