The O.D.E. Method for Convergence of Stochastic Approximation and Reinforcement Learning II: Mean-Square Error under Bounded StepsizesResearch Paper
Stochastic approximation is the family of recursive algorithms that locate a zero of a vector field from noisy evaluations of it. Temporal-difference learning, Q-learning, actor–critic methods and stochastic gradient descent are all instances. In practice these algorithms are often run with a constant or bounded, non-vanishing stepsize: the iterate keeps…