Scalable Fine-tuning from Multiple Data Sources: A First-Order Approximation Approach

Dongyue Li, Ziniu Zhang, Lu Wang +1

We study the problem of fine-tuning a language model (LM) for a target task by optimally using the information from $n$ auxiliary tasks. This problem has broad applications in NLP,…