Bellman's Dynamic Programming IX: Markovian Decision Processes and the Maximal Perron RootTextbook
Chapter XI of Richard Bellman's Dynamic Programming (Princeton University Press, 1957; DOI 10.2307/j.ctv1nxcw0f) studies decision processes whose state is a vector of nonnegative quantities, for example the probabilities that a system is in each of states, or the stocks of commodities, and whose transitions are linear maps chosen stage by stage by a…