Multi-Bellman operator for convergence of $Q$-learning with linear function approximation

Diogo S. Carvalho, Pedro A. Santos, Francisco S. Melo

We study the convergence of $Q$-learning with linear function approximation. Our key contribution is the introduction of a novel multi-Bellman operator that extends the traditional…