Example 7.5: the Bernoulli target process with a beta prior has index ν(α, β) = r(α, β) = α/(α + β)
ProvedAllocationIndices.bernoulli_target_indexbernoulli-banditsgittins-indextarget-processes
Example 7.5. Bernoulli target process, , beta prior with parameters : , the sequence of states starting from is , every state is favourable since , and hence .
Formally: for and , the Gittins index of the Bernoulli target process (from the target is reached with probability , otherwise the state becomes ; reward in and in ) in the state equals .
Preamble
import Definitions.Def_AllocationIndices_Sampling open MeasureTheory ProbabilityTheory BanditAlgorithm
Formal statement
namespace AllocationIndices
/-- **Example 7.5** (p. 182), the Bernoulli target process with `T = 1` and a beta prior with
parameters `α, β > 0`: every state `(α, β)` is favourable, since `α/(α + β) ≥ α/(α + β + n)`,
and hence `ν(α, β) = r(α, β) = α/(α + β)`. -/
theorem bernoulli_target_index {α β : ℝ} (hα : 0 < α) (hβ : 0 < β) {a : ℝ} (ha0 : 0 < a)
(ha1 : a < 1) :
gittinsIndex bernoulliTargetChain bernoulliTargetReward a (Sum.inl (α, β)) =
α / (α + β) := by sorry
end AllocationIndices
Source
Gittins, Glazebrook and Weber, Multi-armed Bandit Allocation Indices, 2nd ed., Wiley 2011, doi:10.1002/9780470980033, §7.3 p. 182, Example 7.5
Human review
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.