Lemma 2.1, p. 8 — for x in a subspace 𝒳 and u = Pu₀, u₀ ∼ N(0, I): E⟨x, u⟩u = x and E⟨x, u⟩² = ‖x‖²
ProvedZORiem.RSGD.lemma_2_1gaussianp2o-batch-pfp2ap2o-gran-per-chapterp2o-plan-paperp2o-v1projectionzeroth-order-optimization
Let be a finite-dimensional real inner product space, a linear subspace of dimension , and the orthogonal projection onto . Let be a standard Gaussian vector in and . Then for every the maps and are integrable and
The projected Gaussian direction thus has identity covariance on ; this is what makes the estimator approximately unbiased for the Riemannian gradient.
Formalization Note The paper writes the expectations as with ; here they are integrals against Mathlib's stdGaussian E, which is that probability measure. Integrability is part of the conclusion, so the identities are not satisfied by a junk value of the Bochner integral.
Preamble
import Mathlib open MeasureTheory ProbabilityTheory open scoped InnerProductSpace
Formal statement
namespace ZORiem.RSGD
theorem lemma_2_1 {E : Type*} [NormedAddCommGroup E] [InnerProductSpace ℝ E]
[FiniteDimensional ℝ E] [MeasurableSpace E] [BorelSpace E]
(X : Submodule ℝ E) (d : ℕ) (hX : Module.finrank ℝ X = d) (x : E) (hx : x ∈ X) :
Integrable (fun u₀ : E => ⟪x, X.starProjection u₀⟫_ℝ • X.starProjection u₀)
(stdGaussian E) ∧
∫ u₀, ⟪x, X.starProjection u₀⟫_ℝ • X.starProjection u₀ ∂(stdGaussian E) = x ∧
Integrable (fun u₀ : E => ⟪x, X.starProjection u₀⟫_ℝ ^ 2) (stdGaussian E) ∧
∫ u₀, ⟪x, X.starProjection u₀⟫_ℝ ^ 2 ∂(stdGaussian E) = ‖x‖ ^ 2 := by sorry
end ZORiem.RSGD
Source
Li, Balasubramanian and Ma, arXiv:2003.11238v3, Lemma 2.1, (2.6), p. 8
Human review
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.