Lemma 4.4: a stoppable bandit process with an improving stopping option satisfies Condition D
ProvedAllocationIndices.stoppable_conditionDcondition-dstoppable-banditssuperprocesses
Lemma 4.4. Condition D holds for a stoppable bandit process with an improving stopping option.
Formally: for a bandit process on a countable state space with bounded reward, a bounded stopping parameter , , such that is almost surely nondecreasing in process time from every initial state, the stoppable bandit process (continuation control: move by and collect ; stop control: state unchanged, collect ) satisfies Condition D. Boundedness of is the standing bounded-rewards assumption applied to the stop control.
Preamble
import Definitions.Def_AllocationIndices_Superprocess open MeasureTheory ProbabilityTheory BanditAlgorithm
Formal statement
namespace AllocationIndices
theorem stoppable_conditionD {S : Type*} [MeasurableSpace S] [Countable S]
[MeasurableSingletonClass S] (P : Kernel S S) [IsMarkovKernel P] {r μ : S → ℝ}
(hr : BoundedReward r) (hμ : BoundedReward μ) {a : ℝ} (ha0 : 0 < a) (ha1 : a < 1)
(himp : HasImprovingStoppingOption P μ) :
ConditionD (stoppable P r μ) a := by sorry
end AllocationIndices
Source
Gittins, Glazebrook and Weber, Multi-armed Bandit Allocation Indices, 2nd ed., Wiley 2011, doi:10.1002/9780470980033, §4.4 p. 88, Lemma 4.4 with its proof on p. 89
Human review
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.