A Critical Look at Targeted Instruction Selection: Disentangling What Matters (and What Doesn't)

Nihal V. Nayak, Paula Rodriguez-Diaz, Neha Hulkund +2

Instruction fine-tuning of large language models (LLMs) often involves selecting a subset of instruction training data from a large candidate pool, using a small query set from the…