PPL-Factory: Task-Aware and Budget-Aware Data Selection from Language Modeling to Reasoning

Hang Zhang, Warren J. Gross

Not all training samples contribute equally to large language model fine-tuning. Selecting informative training samples can reduce the computational cost while preserving downstrea…