Instruction Diversity Drives Generalization To Unseen Tasks

Dylan Zhang, Justin Wang, Francois Charton

Instruction tuning -- fine-tuning a large language model (LLM) on pairs of instructions and desired outcomes -- is an approach that enables pre-trained language models to perform r…