Markov Decision Processes (Puterman) II: Markov Policies Preserve Reward CriteriaTextbook
A controller making repeated decisions can remember every state visited and every action taken. The resulting policy class is much larger than the class of rules that use only the current state and time. In applications modeled by Markov decision processes, understanding whether that extra memory changes achievable expected rewards determines which policy…