Consistent Variance Estimation for Q-Function Estimators in Finite-Horizon MDP Tree Search

Zhenyu Yue, Jie Xu, Chun-Hung Chen +2

We study the variance of Q-function estimators in finite-horizon, finite-state Markov decision process (MDP) tree search. We show that the variance decomposes into three components…