Theorem 10.4.1: , with equality iff and are independent
ProvedWildeQIT.ZhangAudenaert.mutualInfo_nonnegentropyinformation-theoryp2o-gran-per-chapterp2o-plan-bookp2o-v1wilde-qit
Let and be random variables on finite alphabets with joint distribution , and let be their mutual information (Definition 10.4.1, in bits). Then
and if and only if and are independent, that is, for all .
Mutual information measures the correlation between two random variables; it vanishes exactly on product distributions. This is the classical precursor of strong subadditivity (Theorem 10.6.1).
Formalization Note The pair is represented by its joint distribution on a product of finite types; independence is FinDist.Indep.
Differences from the source: none.
Preamble
import Mathlib import Definitions.Def_WildeQIT_mutualInfo
Formal statement
namespace WildeQIT.ZhangAudenaert
/-- **Theorem 10.4.1 (Non-negativity of mutual information).**
Wilde, *From Classical to Quantum Shannon Theory*, arXiv:1106.1445v8, Theorem 10.4.1, (10.30),
p. 296: "The mutual information `I(X;Y)` is non-negative for any random variables `X` and `Y`:
`I(X;Y) ≥ 0`, and `I(X;Y) = 0` if and only if `X` and `Y` are independent random variables
(i.e., if `p_{X,Y}(x,y) = p_X(x) p_Y(y)`)."
**Formalization Note.** The pair `(X, Y)` is its joint distribution `p` on `α × β` (finite
alphabets); `I(X;Y) = mutualInfo p = H(X) − H(X|Y)` (Definition 10.4.1, bits), and
independence is `FinDist.Indep`. -/
theorem mutualInfo_nonneg {α β : Type} [Fintype α] [Fintype β] (p : FinDist (α × β)) :
0 ≤ mutualInfo p ∧ (mutualInfo p = 0 ↔ p.Indep) := by sorry
end WildeQIT.ZhangAudenaert
Source
Wilde, From Classical to Quantum Shannon Theory, arXiv:1106.1445v8, Theorem 10.4.1, eq. (10.30), p. 296
Human review
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.