Approximation Benefits of Policy Gradient Methods with Aggregated States: Every Stationary Point on the State-Aggregated Policy Class Has Per-Period Regret at Most Research Paper
Policy gradient methods optimize a parameterized policy by first-order ascent on the expected discounted reward. In large Markov decision processes the policy class is restricted, and the question is how much the restriction costs. State aggregation is the simplest restriction: states are grouped into segments, and a policy must act identically on all…