Relative entropy between push-forward measures
ProvedInformationTheory.klDiv_map_eq_klDiv_trim_comapObserving a random element and restricting to the -algebra it generates are the same thing, as far as relative entropy is concerned.
Let be finite measures on and let be measurable. Write and for the push-forwards, , and for the -algebra generated by . Then
Both sides measure exactly the information about the hypothesis versus that is carried by the observation : on the left it is packaged as a measure on the observation space, on the right as a measure on the original space seen through a coarser -algebra.
The identity is the standard bridge between the two ways of stating a data-processing or change-of-measure argument, and it is what lets one move a divergence computation from an abstract filtered space to a concrete product space where a chain rule is available.
Formalization Note is MeasurableSpace.comap, and the restriction is Measure.trim. The proof needs no separability or standard-Borel assumption: every -measurable set is by definition a preimage, which is all that is used. The density on is the pull-back .
import Mathlib.InformationTheory.KullbackLeibler.Basic open MeasureTheory InformationTheory Set open scoped ENNReal
theorem InformationTheory.klDiv_map_eq_klDiv_trim_comap {α β : Type*}
{mα : MeasurableSpace α} {mβ : MeasurableSpace β}
{φ : α → β} (hφ : Measurable φ) (μ ν : Measure α)
[IsFiniteMeasure μ] [IsFiniteMeasure ν] :
klDiv (μ.map φ) (ν.map φ) =
@klDiv α (mβ.comap φ) (μ.trim hφ.comap_le) (ν.trim hφ.comap_le) := by
sorry