Deep linear network: first-layer two-point correlator (eq. 3.10)
ProvedDeepLearningTheory.linearNet_two_point_first_layerdeep-learning-theorymachine-learningprobability
For a zero-bias deep linear network at initialization (eq. 3.4), any two inputs and any first-layer neurons ,
Preamble
import Mathlib import Definitions.Def_DLT_DeepLinearNetwork open MeasureTheory ProbabilityTheory open scoped NNReal
Formal statement
namespace DeepLearningTheory
theorem linearNet_two_point_first_layer {Ω : Type*} [MeasurableSpace Ω] (P : Measure Ω)
[IsProbabilityMeasure P] (n : ℕ → ℕ) (CW : ℝ≥0) (W : Ω → ℕ → ℕ → ℕ → ℝ)
(hW : IsLinearNetInit P n CW W) (x₁ x₂ : ℕ → ℝ) (i₁ i₂ : ℕ)
(hi₁ : i₁ < n 1) (hi₂ : i₂ < n 1) :
∫ ω, linearPreact n (W ω) x₁ 1 i₁ * linearPreact n (W ω) x₂ 1 i₂ ∂P
= kron i₁ i₂ * (CW : ℝ) * inputKernel (n 0) x₁ x₂ := by sorry
end DeepLearningTheorySource
Daniel A. Roberts and Sho Yaida (with Boris Hanin), *The Principles of Deep Learning Theory*, arXiv:2106.10165v2, https://arxiv.org/abs/2106.10165, §3.2, p. 56, eqs. (3.8)–(3.10) (page numbers are the book's printed page numbers)
Read-back
What the Lean code literally says, in plain math · Aristotle (Harmonic) - non-blind, same agent as the drafter
Non-blind read-back. This read-back was written by the same agent that drafted the Lean statement, with full knowledge of the source text and of the intended meaning. It is not independent testimony and must not be mistaken for a blind audit; reviewers should compare the Lean code against the source themselves.
Let be a probability space, widths, and satisfying IsLinearNetInit (independent weights, for , , ). For all inputs and all :
where . If both sides are .
Human review
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.