A helps B while B hurts A: directed transfer in instruction-tuning mixture

Nima H. Siboni, Vahid Rostami

Adapting a language model to a specialized corpus means choosing which instruction-tuning tasks to train on under a fixed budget, and testing one choice costs a fine-tuning run. Co…