The projection-updating theorem

Work in a Hilbert space $H$ of random variables, where the inner product of two variables is their correlation, $\langle a, b\rangle = E[ab]$.

Let $\beta \in H$ and let $\hat\beta_1$ be its orthogonal projection on a closed subspace $Y_1$ — the best estimate of $\beta$ given the data generating $Y_1$. Let $y_2$ be an $m$-vector of random variables generating a subspace $Y_2$, let $\hat y_2$ be the vector of projections of the components of $y_2$ onto $Y_1$ (the best estimates of $y_2$ from the old data), and set

Then the projection of $\beta$ onto $Y_1 + Y_2$ is

that is, $\hat\beta_1$ plus the best estimate of $\beta$ in the subspace generated by $\tilde y_2$.

The mechanism is purely geometric: $\tilde y_2$ is orthogonal to $Y_1$ by construction and generates a subspace $\tilde Y_2$ with $Y_1 + Y_2 = Y_1 \oplus \tilde Y_2$, and a projection onto a sum of orthogonal subspaces is the sum of the projections. The old estimate is never recomputed — only the part of the new data that could not be anticipated from the old contributes. This is the engine behind sequential estimation and, in particular, the Kalman recursion.

Formalization Note. As in the source, the statement is given for a single random variable $\beta$; the vector case is $n$ separate problems. The projection onto the enlarged subspace is characterized by membership plus orthogonality of the error, which determines it uniquely, rather than through a projection operator. G is the Gram matrix of the innovations, assumed invertible.