Proposition 1(ii), p. 12 β joint convexity of the relative entropy I(ββ²,β) on π« Γ π«
ProvedDROOptimal.Predictor.proposition_1_iiconvex-analysisinformation-theoryp2o-batch-pfp2ap2o-gran-per-chapterp2o-plan-paperp2o-v1relative-entropy
Let be the probability simplex on and the relative entropy of with respect to (Definition 5, with for ). For all pairs and every ,
The proof of Theorem 4 uses this with , to move a model along a segment while keeping its relative entropy distance from the observed frequencies below .
Formalization Note Both sides are computed in EReal, where ; so at (or ) an infinite term multiplied by zero does not contribute, as in the usual convention of convex analysis.
Preamble
import Mathlib import Definitions.Def_DROOptimal_Predictor_Setting import Definitions.Def_DROOptimal_Predictor_Problem5
Formal statement
namespace DROOptimal.Predictor
/-- Proposition 1(ii) (Convexity), p. 12: for all pairs (ββ²β, ββ), (ββ²β, ββ) β π« Γ π« and
Ξ» β [0, 1],
I((1 β Ξ»)ββ²β + Ξ»ββ²β, (1 β Ξ»)ββ + Ξ»ββ) β€ (1 β Ξ») I(ββ²β, ββ) + Ξ» I(ββ²β, ββ),
in `EReal` (where 0 Β· β = 0). -/
theorem proposition_1_ii {d : β} (β'β ββ β'β ββ : Ξ d) (t : β) (ht : t β Set.Icc (0 : β) 1) :
relEntropy (mix t ht β'β β'β) (mix t ht ββ ββ) β€
((1 - t : β) : EReal) * relEntropy β'β ββ + ((t : β) : EReal) * relEntropy β'β ββ := by sorry
end DROOptimal.Predictor
Source
Van Parys, Mohajerin Esfahani & Kuhn, From Data to Decisions: Distributionally Robust Optimization is Optimal, arXiv:1704.04118v3, p. 12, Proposition 1(ii)
Human review
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.