Rates of Convergence in the Central Limit Theorem for Markov Chains, with an Application to TD Learning: Wasserstein Rate (log n)²/√n for General State-Space Chains under Geometric DriftResearch Paper
Averages along a Markov chain are the basic estimator of Markov chain Monte Carlo, of simulation output analysis, and of reinforcement-learning algorithms such as temporal-difference (TD) learning, whose iterates are driven by a Markov chain of states. The Markov chain central limit theorem says that, after centering and scaling by , such averages…