The Flan Collection: Designing Data and Methods for Effective Instruction Tuning
arXiv:2301.13688
Abstract
We study the design decisions of publicly available instruction tuning methods, and break down the development of Flan 2022 (Chung et al., 2022). Through careful ablation studies on the Flan Collection of tasks and methods, we tease apart the effect of design decisions which enable Flan-T5 to outperform prior work by 3-17%+ across evaluation settings. We find task balancing and enrichment techniques are overlooked but critical to effective instruction tuning, and in particular, training with mixed prompt settings (zero-shot, few-shot, and chain-of-thought) actually yields stronger (2%+) performance in all settings. In further experiments, we show Flan-T5 requires less finetuning to converge higher and faster than T5 on single downstream tasks, motivating instruction-tuned models as more computationally-efficient starting checkpoints for new tasks. Finally, to accelerate research on instruction tuning, we make the Flan 2022 collection of datasets, templates, and methods publicly available at https://github.com/google-research/FLAN/tree/main/flan/v2.
Cited by in corpus (10)
- Adapted Large Language Models Can Outperform Medical Experts in Clinical Text Summarization
- Learning from models beyond fine-tuning
- Evaluating Search Engines and Large Language Models for Answering Health Questions
- RAMIE: Retrieval-Augmented Multi-task Information Extraction with Large Language Models on Dietary Supplements
- Large Language Models for Scholarly Ontology Generation: An Extensive Analysis in the Engineering Field
- Undesirable Biases in NLP: Addressing Challenges of Measurement
- Assessing the Effectiveness of LLMs in Android Application Vulnerability Analysis
- Synthetically generated text for supervised text analysis
- Enhancing chest X-ray datasets with privacy-preserving large language models and multi-type annotations: a data-driven approach for improved classification
- Scalable Multitask Learning Using Gradient-based Estimation of Task Affinity