Showing cs.LGShow all
2 papers · 1 filter
cs.LG2025
Influence Functions for Efficient Data Selection in Reasoning
Prateek Humane, Paolo Cudrano, Daniel Z. Kaplan +3
Fine-tuning large language models (LLMs) on chain-of-thought (CoT) data shows that a small amount of high-quality data can outperform massive datasets. Yet, what constitutes "quali…
cs.LG2025
Training Dynamics Underlying Language Model Scaling Laws: Loss Deceleration and Zero-Sum Learning
Andrei Mircea, Supriyo Chakraborty, Nima Chitsazan +4
This work aims to understand how scaling improves language models, specifically in terms of training dynamics. We find that language models undergo loss deceleration early in train…