Offline Estimation of Controlled Markov Chains: Minimaxity and Sample Complexity

Imon Banerjee, Harsha Honnappa, Vinayak Rao

In this work, we study a natural nonparametric estimator of the transition probability matrices of a finite controlled Markov chain. We consider an offline setting with a fixed dat…