Integrability of conditional mutual information with a finite coordinate
ProvedInformationTheory.conditional_finite_mutualInformation_integrablebanditsconditional-mutual-informationinformation-theoryprobability
Let be a probability measure, let and be standard Borel spaces, and let take values in the finite set . For measurable random variables , , and , write , , and for their regular conditional laws given . Then the conditional mutual-information density is integrable:
This finite-coordinate result supplies the measurability and integrability needed to use conditional information gains in information-ratio arguments.
Formalization Note The proof represents the density as an integral of categorical posterior divergences and bounds it by the entropy of a distribution on .
Preamble
import Mathlib.Probability.Kernel.CondDistrib import Mathlib.InformationTheory.KullbackLeibler.Basic open MeasureTheory ProbabilityTheory InformationTheory
Formal statement
namespace InformationTheory
/-- Conditional mutual information is integrable when one coordinate takes
values in a finite type. -/
theorem conditional_finite_mutualInformation_integrable
{Omega H Y : Type} {mOmega : MeasurableSpace Omega}
{mH : MeasurableSpace H} {mY : MeasurableSpace Y}
[StandardBorelSpace H] [StandardBorelSpace Y] [Nonempty Y]
{k : ℕ} [NeZero k]
(mu : Measure Omega) [IsProbabilityMeasure mu]
(hist : Omega → H) (obs : Omega → Y) (act : Omega → Fin k)
(hhist : Measurable hist) (hobs : Measurable obs) (hact : Measurable act) :
Integrable
(fun h ↦ (klDiv (condDistrib (fun w ↦ (act w, obs w)) hist mu h)
((condDistrib act hist mu h).prod (condDistrib obs hist mu h))).toReal)
(mu.map hist) := by
sorry
end InformationTheorySource
Lattimore and Szepesvari, Bandit Algorithms (2020), Lemma 36.7, printed p. 470 / free PDF p. 479; measure-theoretic finite-posterior integrability bridge used to formalize its conditional mutual information.