Learning Markov Models Via Low-Rank Optimization: KL Error Bound for the Rank-Constrained Maximum Likelihood Estimator of a Low-Rank Markov ChainResearch Paper
Many systems studied in operations research and reinforcement learning are modelled as finite Markov chains with a large number of states : the stops of a taxi in a city, a random walk on a network, the conformations of a molecule. Estimating the transition matrix from one observed trajectory needs, in general, a trajectory long compared with the …