4 papers
Timestep Weighting: A Hidden Key to Effective ELBO-Based Flow-Matching RL
Qinwei Ma, Jingzhe Shi, Simin Fan +3
ELBO-based reinforcement learning offers a sampler-agnostic approach to fine-tuning flow matching models with reward feedback. Timestep weighting in ELBO-based RL has large impact…
The Magic Correlations: Understanding Knowledge Transfer from Pretraining to Supervised Fine-Tuning
Simin Fan, Dimitris Paparas, Natasha Noy +3
Understanding how language model capabilities transfer from pretraining to supervised fine-tuning (SFT) is fundamental to efficient model development and data curation. In this wor…
Dynamic Gradient Alignment for Online Data Mixing
Simin Fan, David Grangier, Pierre Ablin
The composition of training data mixtures is critical for effectively training large language models (LLMs), as it directly impacts their performance on downstream tasks. Our goal…
Task-Adaptive Pretrained Language Models via Clustered-Importance Sampling
David Grangier, Simin Fan, Skyler Seto +1
Specialist language models (LMs) focus on a specific task or domain on which they often outperform generalist LMs of the same size. However, the specialist data needed to pretrain…