2 papers
cs.CL2026
Synthetic Data for any Differentiable Target
Tristan Thrush, Sung Min Park, Herman Brunborg +5
What are the limits of controlling language models via synthetic training data? We develop a reinforcement learning (RL) primitive, the Dataset Policy Gradient (DPG), which can pre…
cs.CL2025
Improving Pretraining Data Using Perplexity Correlations
Tristan Thrush, Christopher Potts, Tatsunori Hashimoto
Quality pretraining data is often seen as the key to high-performance language models. However, progress in understanding pretraining data has been slow due to the costly pretraini…