Read-back: decomposition_error_sup_bound

What the statement literally asserts

Fix a natural number $N$ (no positivity assumed) and, for each index $i \in \{0,\dots,N-1\}$, a type $\mathcal S_i$ that is assumed finite and to have decidable equality. Write

$$\Pi \;=\; \prod_{i} \mathcal S_i$$

for the joint space, whose elements are the dependent tuples $p = (p_i)_i$ with $p_i \in \mathcal S_i$; $\Pi$ is itself finite. The statement then takes as data: a real matrix $P$ indexed by $\Pi \times \Pi$; a real-valued function $\mu$ on $\Pi$; a real number $\gamma$; a function $r_{\max} : \{0,\dots,N-1\} \to \mathbb R$; for each $i$ a real-valued function $r_i$ on $\mathcal S_i$; a real-valued function $Q$ on $\Pi$; for each $i$ a real matrix $P^{(i)}$ indexed by $\mathcal S_i \times \mathcal S_i$; and for each $i$ a real-valued function $Q_i$ on $\mathcal S_i$. It assumes:

1. $0 \le \gamma$ and $\gamma < 1$ (so $\gamma \in [0,1)$; note $\gamma = 0$ is allowed).
2. $P$ is a transition matrix: every entry satisfies $P_{p q} \ge 0$, and every row sums to one, $\sum_{q \in \Pi} P_{p q} = 1$ for all $p$.
3. $\mu$ is a strictly positive distribution: $\mu(p) > 0$ for every $p \in \Pi$, and $\sum_{p \in \Pi} \mu(p) = 1$.
4. $\mu$ is stationary for $P$: $\sum_{p \in \Pi} \mu(p) P_{p q} = \mu(q)$ for every $q \in \Pi$.
5. Reward bound: $|r_i(s)| \le r_{\max}(i)$ for every $i$ and every $s \in \mathcal S_i$.
6. $Q$ is a Bellman fixed point on the joint chain with the additive reward: for every $p \in \Pi$,

$$Q(p) \;=\; \Big(\textstyle\sum_i r_i(p_i)\Big) \;+\; \gamma \sum_{q \in \Pi} P_{p q}\, Q(q).$$

(The joint reward here is by construction the sum of the same local rewards $r_i$ used below; it is not a free joint reward function.) 7. Each $P^{(i)}$ is a transition matrix: nonnegative entries, rows summing to one on $\mathcal S_i$. 8. Each $P^{(i)}$ exactly attains the agent-$i$ entanglement infimum: $\operatorname{muAgentTV}_i(\mu, P, P^{(i)}) = E_i(\mu, P)$, where the two sides unfold as follows.

• The marginal one-step law of agent $i$ out of a joint point $p$ is

$$m_i(P, p, t) \;=\; \sum_{q \in \Pi} \big[\,q_i = t\,\big]\, P_{p q}, \qquad t \in \mathcal S_i,$$

 i.e. the joint row at $p$ summed over all coordinates other than $i$.

• The $\mu$-weighted agent-wise total variation distance from $P$ to a candidate local matrix $M$ on $\mathcal S_i$ is

$$\operatorname{muAgentTV}_i(\mu, P, M) \;=\; \sum_{p \in \Pi} \mu(p)\cdot \tfrac12 \sum_{t \in \mathcal S_i} \big| m_i(P,p,t) - M_{p_i\, t} \big|.$$

 (It is a $\mu$-weighted *average* over joint points, not a maximum.)

• The agent-$i$ measure of Markov entanglement is the infimum of that quantity over all local transition matrices:

$$E_i(\mu, P) \;=\; \inf \big\{\, c \in \mathbb R \;:\; \exists\, M \text{ a transition matrix on } \mathcal S_i,\ c = \operatorname{muAgentTV}_i(\mu, P, M) \,\big\}.$$

 (This is a set-infimum of reals; the underlying set is nonempty and bounded below by $0$ under the hypotheses, so no junk value arises. Note that $E_i$ is defined by comparison with a *single* local matrix per agent; the separability notion `IsSeparableN` and the abstract `entanglementWith` from the same bundle are **not** used anywhere in this statement.)

So hypothesis 8 says each $P^{(i)}$ is an exact minimiser, not merely an approximate one. 9. Each $Q_i$ is a Bellman fixed point of the local chain with local reward $r_i$: for every $s \in \mathcal S_i$,

$$Q_i(s) \;=\; r_i(s) \;+\; \gamma \sum_{t \in \mathcal S_i} P^{(i)}_{s t}\, Q_i(t).$$

10. A joint point $p \in \Pi$ is given (an ordinary universally quantified argument).

Under all of the above, the conclusion is the pointwise inequality

$$\Big| \, Q(p) \;-\; \sum_i Q_i(p_i) \, \Big| \;\;\le\;\; \frac{4\,\gamma \sum_i E_i(\mu, P)\, r_{\max}(i)}{(1-\gamma)^2}.$$

The inequality is non-strict ($\le$). Because $p$ is a bound argument, the assertion is "for every joint point $p$", i.e. the bound applies uniformly; but the left side is the plain absolute value at $p$, not a supremum, a norm, or a $\mu$-weighted average. Nothing about separability of $P$, product form of $P$, or any relation between $P$ and the matrices $P^{(i)}$ beyond hypothesis 8 is assumed. In particular $P^{(i)}$ is not required to satisfy the bundle's IsLocalTransitionN (the marginalisation identity), only to be a transition matrix attaining the infimum.

Status of every variable occurring in the conclusion

• $p$ — entirely free: it ranges over all of $\Pi$, with no constraint whatsoever. The claim must hold at every joint point simultaneously.
• $Q$ — constrained, and in fact pinned down: hypothesis 6 is the Bellman equation for the joint chain, which for $\gamma \in [0,1)$ and a row-stochastic $P$ has exactly one solution. $Q$ is therefore determined by $P$, $\gamma$ and the $r_i$.
• $Q_i$ — likewise constrained and pinned down by hypothesis 9, determined by $P^{(i)}$, $\gamma$, $r_i$.
• $\gamma$ — constrained to $[0,1)$ by hypotheses 1; free within that range. Since $\gamma < 1$ strictly, $(1-\gamma)^2 > 0$ and the division on the right is never a division by zero (no junk value from division). At $\gamma = 0$ the right side is exactly $0$ and the claim becomes the exact identity $Q(p) = \sum_i Q_i(p_i)$ (which at $\gamma = 0$ reads $\sum_i r_i(p_i) = \sum_i r_i(p_i)$).
• $r_{\max}$ — constrained only from below, by hypothesis 5: $r_{\max}(i) \ge \sup_{s} |r_i(s)|$. It is otherwise free and may be arbitrarily large. Because $E_i(\mu,P) \ge 0$ and $\gamma \ge 0$, enlarging $r_{\max}$ only enlarges the right-hand side, so the binding instance of the claim is $r_{\max}(i) = \max_s |r_i(s)|$. $r_{\max}(i)$ cannot be negative under the hypotheses, because $\mathcal S_i$ is forced to be nonempty (see below).
• $\mu$ and $P$ — constrained by hypotheses 2–4; they enter the conclusion only through the numbers $E_i(\mu, P)$. $\mu$'s stationarity and strict positivity are assumed but the conclusion refers to $\mu$ nowhere else.
• $P^{(i)}$ — constrained by hypotheses 7–8; it does not appear in the conclusion directly, only through $Q_i$.
• $r_i$ — free functions, constrained only through hypothesis 5 and through their appearance in both Bellman equations 6 and 9. Note that the joint reward in hypothesis 6 and the local rewards in hypothesis 9 are the same family $r_i$; the joint reward is additive by construction.
• $N$, $\mathcal S_i$ — free (implicit) parameters; the finiteness and decidable-equality instances are the only assumptions on them.

Degenerate and edge cases

• $N = 0$. Then $\Pi$ has exactly one element (the empty tuple), all sums $\sum_i(\cdot)$ over agents are empty and equal $0$, and $P$ is the $1\times 1$ matrix $[1]$, $\mu = 1$. The Bellman equation forces $Q = \gamma Q$, hence $Q = 0$; both sides of the conclusion are $0$ and it holds with equality. The statement does not exclude this case.
• Some $\mathcal S_i$ empty. Then $\Pi$ is empty and $\sum_{p \in \Pi}\mu(p) = 0 \ne 1$, so hypothesis 3 is contradictory. Consequently the hypotheses silently force every $\mathcal S_i$ to be nonempty (which in turn forces $r_{\max}(i) \ge 0$).
• Vanishing entanglement. If $E_i(\mu, P) = 0$ for every $i$ (which happens, for instance, whenever $N \le 1$, or whenever each agent's marginal one-step law depends only on that agent's own coordinate), the right side is exactly $0$ and the conclusion asserts the exact decomposition $Q(p) = \sum_i Q_i(p_i)$ for every $p$.
• Infimum conventions. $E_i(\mu, P)$ is a real-number infimum of a set; under the hypotheses that set is nonempty (transition matrices on the nonempty finite $\mathcal S_i$ exist) and bounded below by $0$ (since $\mu \ge 0$), so $E_i(\mu,P) \ge 0$ and the usual convention "infimum of an empty or unbounded set is $0$" is never invoked here.
• No Nat subtraction, no division by a possibly-zero quantity, and no other total-function junk values occur in the statement.

Joint satisfiability of the hypotheses

The hypotheses are jointly satisfiable, so the statement is not vacuous. A concrete witness: take any $N$, any nonempty finite $\mathcal S_i$, any row-stochastic $P$ on $\Pi$ admitting a strictly positive stationary distribution $\mu$ (e.g. $P$ uniform, $\mu$ uniform), any $\gamma \in [0,1)$, any local rewards $r_i$ with $r_{\max}(i) = \max_s |r_i(s)|$, and let $Q$, $Q_i$ be the unique Bellman solutions. Hypothesis 8 is satisfiable for every such $P, \mu$: the set of transition matrices on the finite nonempty $\mathcal S_i$ is a nonempty compact set and $M \mapsto \operatorname{muAgentTV}_i(\mu, P, M)$ is continuous, so the infimum $E_i(\mu,P)$ is attained by some $P^{(i)}$. Hence for essentially arbitrary $P$, $\mu$, $\gamma$ and $r$ the full hypothesis set can be met, including instances where $E_i(\mu, P) > 0$.

Falsifiability by free choices

Every quantity appearing in the conclusion other than $p$ and $r_{\max}$ is pinned down by the hypotheses, and the two genuinely free ones move the claim in known directions: $p$ is fully universally quantified (so the conclusion is a claim at every joint point, the strongest pointwise form), while $r_{\max}$ can only be increased above $\max_s |r_i(s)|$, which weakens the right-hand side. The one structural point an auditor should note is that the joint reward in hypothesis 6 is forced to be $\sum_i r_i(p_i)$ using the same $r_i$ that drive the local Bellman equations in hypothesis 9, so there is no freedom to choose local rewards unrelated to the joint reward.