Global Optimality Guarantees for Policy Gradient Methods 6: In the KL-Regularized Finite MDP, a Regularized-Optimal Policy Is Within λ(1 + log(1 + c/λ)) of OptimalResearch Paper
Policy gradient methods for Markov decision processes are frequently run on a regularized objective rather than on the cost of interest: a convex penalty on the action distribution is added to every single-period cost. Regularization smooths the problem, makes optimal policies strictly stochastic, and keeps iterates away from the boundary of the probability…