Markov decision processes: on the convergence of the Monte-Carlo first visit algorithm

Sylvain Delattre, Nicolas Fournier

We consider the Monte-Carlo first visit algorithm, of which the goal is to find the optimal control in a Markov decision process with finite state space and finite number of possib…