Lemma 5.3 — the total variation between two softmax distributions is at most a quarter of the sup-distance of their logits
DisprovedLogitRankDistill.lemma_5_3distillationlanguage-modelslogitsp2o-gran-per-chapterp2o-plan-paperp2o-v1
For vectors let and . Then
This is the Lipschitz estimate the paper uses to pass from logit errors to total variation in the proof of Theorem 5.11 (which needs only ). As printed the constant is false (two tokens, , give ); it is stated as printed and the finding is recorded separately.
Formalization Note. is ⨆ y, |L y - L' y| over the finite type; is .
Differences from the source: none.
Preamble
import Mathlib import Definitions.Def_IsingLTL_BeliefProp_TVDist import Definitions.Def_LogitRankDistill_Models
Formal statement
namespace LogitRankDistill
/-- **Lemma 5.3** (p. 23): for `L, L' ∈ ℝ^Σ`,
`D_TV(softmax L, softmax L') ≤ (1/4) ‖L − L'‖_∞`. (False as printed: see the mission's
INSIGHTS, finding I1.) -/
theorem lemma_5_3 {Tok : Type*} [Fintype Tok] [DecidableEq Tok] (L L' : Tok → ℝ) :
IsingLTL.BeliefProp.tvDist (softmax L) (softmax L') ≤ (1 / 4) * ⨆ y, |L y - L' y| := by sorry
end LogitRankDistill
Source
Golowich, Liu, Shetty, Provably Distilling from Modern Language Models via Low Logit Rank, manuscript of November 3, 2026 (arXiv:2512.09892; sha256 2ce0606e…), p. 23, Lemma 5.3
Human review
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.