Bridging the Training-Inference Gap in LLMs by Leveraging Self-Generated Tokens

Zhepeng Cen, Yao Liu, Siliang Zeng +4

Language models are often trained to maximize the likelihood of the next token given past tokens in the training dataset. However, during inference time, they are utilized differen…