Hellinger affinity vs. KL:
ProvedBanditAlgorithm.kl_hellinger_affinity_boundinformation-theoryprobability
For probability measures P and Q with finite relative entropy, their squared Hellinger affinity is bounded below by the exponential of minus the KL divergence: , where ν=P+Q. The finite-divergence hypothesis is required because ENNReal.toReal uses a junk value at infinity.
Preamble
import Mathlib.InformationTheory.KullbackLeibler.Basic open MeasureTheory InformationTheory open scoped ENNReal
Formal statement
theorem BanditAlgorithm.kl_hellinger_affinity_bound {Ω : Type} {mΩ : MeasurableSpace Ω}
(P Q : Measure Ω) [IsProbabilityMeasure P] [IsProbabilityMeasure Q]
(hD : klDiv P Q ≠ ∞) :
ENNReal.ofReal (Real.exp (-(klDiv P Q).toReal)) ≤
(∫⁻ ω, (P.rnDeriv (P + Q) ω * Q.rnDeriv (P + Q) ω) ^ (2⁻¹ : ℝ)
∂(P + Q)) ^ 2 := by sorrySource
Lattimore--Szepesvari, Bandit Algorithms (CUP 2020), proof of Theorem 14.2, Jensen/KL display immediately after Eq. (14.9), printed p. 191 / PDF p. 200.