Gibbs variational identity at the optimiser
ProvedDonskerVaradhan.klDiv_tilted_self_eqDonsker–Varadhan / Gibbs variational identity (value at the optimizer). Let be a probability measure on and a function with integrable with respect to , and assume is integrable against the tilted (Gibbs) measure whose density with respect to is . Then the Kullback–Leibler divergence of the tilt satisfies
This is the value attained by the Donsker–Varadhan supremum at the optimizer , namely once rearranged. Boucheron–Lugosi–Massart, Concentration Inequalities (OUP 2013), Theorem 4.13 / Corollary 4.14.
Preamble
import Mathlib.InformationTheory.KullbackLeibler.Basic import Mathlib.MeasureTheory.Measure.Tilted open Real MeasureTheory Set open scoped ENNReal NNReal
Formal statement
theorem DonskerVaradhan.klDiv_tilted_self_eq {α : Type*} {mα : MeasurableSpace α} {μ : Measure α} {f : α → ℝ} [IsProbabilityMeasure μ] (hf : Integrable (fun x => exp (f x)) μ) (hf_int : Integrable f (μ.tilted f)) : (InformationTheory.klDiv (μ.tilted f) μ).toReal = ∫ x, f x ∂(μ.tilted f) - log (∫ x, exp (f x) ∂μ) := by sorryHuman review
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.