KL divergence between two real Gaussians of equal variance
ProvedBanditAlgorithm.klDiv_gaussianReal_same_variancegaussianinformation-theorykullback-leibler
The Kullback--Leibler divergence between two Gaussians of the same variance is
This closed form is absent from Mathlib and is the arithmetic backbone of the whole Gaussian best-arm-identification chapter: it turns the information-theoretic characteristic time into the quadratic optimisation over means that Lattimore--Szepesv'ari solve in closed form. The proof computes the log-likelihood ratio , which is affine in , so only the first moment of is needed.
Preamble
import Mathlib.Probability.Distributions.Gaussian.Real import Mathlib.InformationTheory.KullbackLeibler.Basic open MeasureTheory ProbabilityTheory InformationTheory NNReal ENNReal
Formal statement
theorem BanditAlgorithm.klDiv_gaussianReal_same_variance {v : NNReal} (hv : v ≠ 0) (a b : ℝ) :
InformationTheory.klDiv (ProbabilityTheory.gaussianReal a v)
(ProbabilityTheory.gaussianReal b v)
= ENNReal.ofReal ((a - b) ^ 2 / (2 * v)) := by
sorrySource
Standard; see e.g. Lattimore & Szepesvari, Bandit Algorithms (CUP 2020), Section 33.1 (the Gaussian class E^k_N(1)), and Cover & Thomas, Elements of Information Theory, Chapter 8.