The chain's conditional expectation given the past is one step of the kernel

The Markov property as an identity of conditional expectations. For a bounded measurable $h$ and every $k$,

where $(Ph)(x) = \int h\,dP(x,\cdot)$ and the conditioning $\sigma$-algebra is the pullback of the Borel structure on the first $k+1$ coordinates. This holds for every initial distribution.

What it is for. This is the hypothesis that turns Poisson-equation bookkeeping into a genuine martingale. If $g$ solves $g - Pg = \varphi - \mathbb E_\pi\varphi$ then

satisfies $\mathbb E[D_k \mid \sigma(X_0,\dots,X_k)] = 0$, i.e. $(D_k)$ is a martingale difference sequence for the natural filtration of the chain — exactly the input required by every martingale central limit theorem. It also gives orthogonality, $\mathbb E[D_jD_k] = 0$ for $j\ne k$, hence $\operatorname{Var}\bigl(\sum_{k<n}D_k\bigr) = \sum_{k<n}\mathbb E D_k^2$, the linear-in-$n$ variance bound behind the law of large numbers for the quadratic variation.

Proof. The candidate $(Ph)(X_k)$ is measurable with respect to $\sigma(X_0,\dots,X_k)$ — the coordinate map $\omega\mapsto\omega_k$ factors through the restriction to the first $k+1$ coordinates, and $x\mapsto\int h\,dP(x,\cdot)$ is measurable because integration against a kernel preserves measurability. It is integrable, being bounded by $\|h\|_\infty$. Finally, for every event $A$ in the conditioning $\sigma$-algebra the two set integrals agree,

which is the Markov property in integral form. The standard characterization of conditional expectation — a measurable, integrable function with the correct set integrals is the conditional expectation — then gives the claim.