On the Complexity of Offline Reinforcement Learning with $Q^{\star }$-Approximation and Partial Coverage

Haolin Liu, Braham Snyder, Chen-Yu Wei

We study offline reinforcement learning under $Q^{\star }$-approximation and partial coverage, a setting that motivates practical algorithms such as Conservative $Q$-Learning (CQL; Ku…