Influence Functions for Efficient Data Selection in Reasoning

Prateek Humane, Paolo Cudrano, Daniel Z. Kaplan +3

Fine-tuning large language models (LLMs) on chain-of-thought (CoT) data shows that a small amount of high-quality data can outperform massive datasets. Yet, what constitutes "quali…