Unifying Question Answering, Text Classification, and Regression via Span Extraction

Abstract

Even as pre-trained language encoders such as BERT are shared across many tasks, the output layers of question answering, text classification, and regression models are significantly different. Span decoders are frequently used for question answering, fixed-class, classification layers for text classification, and similarity-scoring layers for regression tasks, We show that this distinction is not necessary and that all three can be unified as span extraction. A unified, span-extraction approach leads to superior or comparable performance in supplementary supervised pre-trained, low-data, and multi-task learning experiments on several question answering, text classification, and regression benchmarks.

updating paper to also include regression tasks

References in corpus (6)

• Natural Language Processing (almost) from Scratch
• The Natural Language Decathlon: Multitask Learning as Question Answering
• CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge
• Sentence Encoders on STILTs: Supplementary Training on Intermediate Labeled-data Tasks
• Multi-Task Deep Neural Networks for Natural Language Understanding
• XNLI: Evaluating Cross-lingual Sentence Representations

Cited by in corpus (9)

• CTRL: A Conditional Transformer Language Model for Controllable Generation
• mT5: A massively multilingual pre-trained text-to-text transformer
• Machine Reading Comprehension: The Role of Contextualized Language Models and Beyond
• Unsupervised Pre-training for Biomedical Question Answering
• Probing and Fine-tuning Reading Comprehension Models for Few-shot Event Extraction
• HUBERT Untangles BERT to Improve Transfer across NLP Tasks
• Zero-shot Generalization in Dialog State Tracking through Generative Question Answering
• Unsupervised Multiple Choices Question Answering: Start Learning from Basic Knowledge
• MaP: A Matrix-based Prediction Approach to Improve Span Extraction in Machine Reading Comprehension