Uncertainty Quantification and Exploration for Reinforcement Learning: Empirical Q-Values Are Strongly Consistent and √n(Q̂ₙ − Q) ⇒ N(0, Σ) with the Closed-Form Covariance (5)Research Paper
A reinforcement-learning agent that learns a Markov decision process (MDP) from data ends up with point estimates of the Q-values: the long-run discounted reward of taking an action in a state and acting optimally afterwards. Point estimates say nothing about their own reliability. Choosing between two actions whose estimated values differ by a small…