Deep linear network: exact four-point correlator at initialization (eqs. 3.21, 3.25)
ProvedDeepLearningTheory.linearNet_four_pointExact four-point correlator of a deep linear network. Consider a zero-bias deep linear network with positive widths whose weights are initialized independently with (eq. 3.4). For a single input , every layer and neurons of that layer,
where .
In the infinite-width limit the bracket tends to and the correlator becomes the Gaussian (Wick) value; at finite width the deviation grows with depth, controlled by the depth-to-width ratio.
import Mathlib import Definitions.Def_DLT_DeepLinearNetwork open MeasureTheory ProbabilityTheory open scoped NNReal
namespace DeepLearningTheory
theorem linearNet_four_point {Ω : Type*} [MeasurableSpace Ω] (P : Measure Ω)
[IsProbabilityMeasure P] (n : ℕ → ℕ) (hn : ∀ ℓ, 0 < n ℓ) (CW : ℝ≥0)
(W : Ω → ℕ → ℕ → ℕ → ℝ) (hW : IsLinearNetInit P n CW W) (x : ℕ → ℝ)
(ℓ i₁ i₂ i₃ i₄ : ℕ) (hℓ : 1 ≤ ℓ)
(hi₁ : i₁ < n ℓ) (hi₂ : i₂ < n ℓ) (hi₃ : i₃ < n ℓ) (hi₄ : i₄ < n ℓ) :
∫ ω, linearPreact n (W ω) x ℓ i₁ * linearPreact n (W ω) x ℓ i₂ *
linearPreact n (W ω) x ℓ i₃ * linearPreact n (W ω) x ℓ i₄ ∂P
= wickDelta4 i₁ i₂ i₃ i₄ *
((CW : ℝ) ^ (2 * ℓ) * (∏ ℓ' ∈ Finset.Ico 1 ℓ, (1 + 2 / (n ℓ' : ℝ))) *
inputKernel (n 0) x x ^ 2) := by sorry
end DeepLearningTheoryRead-back
What the Lean code literally says, in plain math · Aristotle (Harmonic) - non-blind, same agent as the drafter
Non-blind read-back. This read-back was written by the same agent that drafted the Lean statement, with full knowledge of the source text and of the intended meaning. It is not independent testimony and must not be mistaken for a blind audit; reviewers should compare the Lean code against the source themselves.
Let be a probability space, widths with for all , , and a random weight field satisfying IsLinearNetInit (the weights , , , , mutually independent with laws ). For every input , every and all :
where , , , and the integral is a Bochner integral.
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.