Change-of-measure pigeonhole with separate deficit and penalty totals
ProvedBanditAlgorithm.change_of_measure_deficit_pigeonhole_two_totalsLet be a finite index set of cardinality , and let with , and . Suppose that for every
Then some index satisfies
This is the change-of-measure summation step of Lattimore--Szepesv'ari, Bandit Algorithms, section 38.7 (eq. 38.24), with the deficit and the Pinsker penalty carried by two different count vectors. In the intended application the observable fed to the bounded change-of-measure inequality is a truncated count, because Step 2 needs it bounded by rather than by ; the divergence, however, is controlled by the full count of the -th pair, since the two Markov decision processes differ in that pair's transition row at every round. Hence the deficit uses the truncated counts, whose total is bounded below, while the Cauchy--Schwarz penalty uses the full counts, whose total is only bounded above. The single-total form is the special case , .
import Mathlib.Data.Real.Sqrt import Mathlib.Algebra.Order.Chebyshev open Finset
theorem BanditAlgorithm.change_of_measure_deficit_pigeonhole_two_totals
{ι : Type*} [Fintype ι] {k : ℕ} (hk : 0 < k) (hcard : Fintype.card ι = k)
(V Vt W : ι → ℝ) (T0 T0full B : ℝ)
(hV : ∀ j, 0 ≤ V j) (hsumV : ∑ j, V j = T0full) (hsumVt : ∑ j, Vt j = T0)
(hB : 0 ≤ B)
(hW : ∀ j, T0 - Vt j - B * Real.sqrt (2 * V j) ≤ W j) :
∃ j : ι,
(((k : ℝ) - 1) * T0 - B * Real.sqrt (2 * k * T0full)) / k ≤ W j := by sorry