Donsker–Varadhan variational inequality
ProvedDonskerVaradhan.integral_sub_klDiv_le_log_integral_expDonsker–Varadhan variational inequality. For probability measures on with integrable log-likelihood ratio, and with integrable against and integrable against , one has
Together with the equality at the tilt , this is the full Donsker–Varadhan formula . The proof is Gibbs' inequality expanded through the tilted log-likelihood ratio. Boucheron–Lugosi–Massart, Concentration Inequalities (OUP 2013), Theorem 4.13 / Corollary 4.14.
Preamble
import Mathlib.InformationTheory.KullbackLeibler.Basic import Mathlib.MeasureTheory.Measure.Tilted open Real MeasureTheory Set open scoped ENNReal NNReal
Formal statement
theorem DonskerVaradhan.integral_sub_klDiv_le_log_integral_exp {α : Type*} {mα : MeasurableSpace α} {μ : Measure α} {f : α → ℝ} {Q : Measure α} [IsProbabilityMeasure μ] [IsProbabilityMeasure Q] (hQμ : Q ≪ μ) (hf : Integrable (fun x => exp (f x)) μ) (hf_int : Integrable f Q) (h_llr : Integrable (llr Q μ) Q) : ∫ x, f x ∂Q - (InformationTheory.klDiv Q μ).toReal ≤ log (∫ x, exp (f x) ∂μ) := by sorryHuman review
Confirmed by the mission captain (proposal self-audit).
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.