Supervised Optimism Correction: Be Confident When LLMs Are Sure

Junjie Zhang, Rushuai Yang, Shunyu Liu +5

In this work, we establish a novel theoretical connection between supervised fine-tuning and offline reinforcement learning under the token-level Markov decision process, revealing…