Eq. (5.5) — the approximations in policy space decrease monotonically

Let $N = n + 1 \ge 2$ and $t_{ij} > 0$ for $i \ne j$, and let $f^{(k)}$ be defined by (5.1) from the direct-route policy (5.2), with $f_N^{(0)} = 0$. Then

The monotone decrease is the "approximation in policy space" property: each iterate is the value of a policy no worse than the previous one.

Formalization Note $f_N^{(0)} = 0$ is the corrected reading of (5.2); see the (5.4) item. With the printed $f_N^{(0)} = t_{NN} > 0$ the inequality fails already at $k = 0$.