Dynamic Programming Deconstructed: Transformations of the Bellman Equation and Computational Efficiency: Refactored Value Iteration Converges to g* and g*-Greedy Policies Are OptimalResearch Paper
Many dynamic programs are solved faster after the Bellman equation has been rewritten. The Q-factor form of reinforcement learning moves the maximization inside an expectation over state-action pairs; Rust's expected value function for discrete choice models (Rust 1987) integrates out the next state before maximizing; optimal stopping problems are routinely…