Deep linear network: vanishing mean preactivation (eq. 3.6)
ProvedDeepLearningTheory.linearNet_mean_zeroConsider a zero-bias deep linear network with widths whose weights are initialized independently with (eq. 3.4). For every input , every layer and every neuron in that layer,
import Mathlib import Definitions.Def_DLT_DeepLinearNetwork open MeasureTheory ProbabilityTheory open scoped NNReal
namespace DeepLearningTheory
theorem linearNet_mean_zero {Ω : Type*} [MeasurableSpace Ω] (P : Measure Ω)
[IsProbabilityMeasure P] (n : ℕ → ℕ) (CW : ℝ≥0) (W : Ω → ℕ → ℕ → ℕ → ℝ)
(hW : IsLinearNetInit P n CW W) (x : ℕ → ℝ) (ℓ i : ℕ) (hℓ : 1 ≤ ℓ) (hi : i < n ℓ) :
∫ ω, linearPreact n (W ω) x ℓ i ∂P = 0 := by sorry
end DeepLearningTheoryRead-back
What the Lean code literally says, in plain math · Aristotle (Harmonic) - non-blind, same agent as the drafter
Non-blind read-back. This read-back was written by the same agent that drafted the Lean statement, with full knowledge of the source text and of the intended meaning. It is not independent testimony and must not be mistaken for a blind audit; reviewers should compare the Lean code against the source themselves.
Let be a probability space, widths, , and a random weight field satisfying IsLinearNetInit (all weights , , , , mutually independent, each with law ). Then for every input , every and every : , where is the zero-bias linear preactivation , evaluated at . Widths may be zero (then some sums are empty). The integral is a Bochner integral.
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.