Mid-Training with Self-Generated Data Improves Reinforcement Learning in Language Models

Aswin RRV, Jacob Dineen, Divij Handa +4

The effectiveness of Reinforcement Learning (RL) in Large Language Models (LLMs) depends on the nature and diversity of the data used before and during RL. In particular, reasoning…