DRIFT: Data Selection for LLM Instruction Tuning via On-Policy Attribution

Zefan Wang, Lincheng Li, Tianyu Yu +1

Data selection for instruction tuning is usually studied for efficiency, reaching strong performance with a fraction of the training data. We ask whether data selection can instead…