Reusing Historical Trajectories in Natural Policy Gradient via Importance Sampling: Asymptotic Normality of the RNPG Error, with Reuse Shrinking the Joint-Noise Covariance by 1/KResearch Paper
In reinforcement learning, each policy update may require new interaction with an environment. When interaction is costly, an optimizer can retain past trajectories and use them again, provided it corrects for the different policy that generated them. Lin, Wang, and Zhou study this idea for natural policy gradient, using importance weights to reuse earlier…