Empirical Evaluation of Multi-task Learning in Deep Neural Networks for Natural Language Processing
arXiv:1908.07820
Abstract
Multi-Task Learning (MTL) aims at boosting the overall performance of each individual task by leveraging useful information contained in multiple related tasks. It has shown great success in natural language processing (NLP). Currently, a number of MLT architectures and learning mechanisms have been proposed for various NLP tasks. However, there is no systematic exploration and comparison of different MLT architectures and learning mechanisms for their strong performance in-depth. In this paper, we conduct a thorough examination of typical MTL methods on a broad range of representative NLP tasks. Our primary goal is to understand the merits and demerits of existing MTL methods in NLP tasks, thus devising new hybrid architectures intended to combine their strengths.
References in corpus (14)
- An Overview of Multi-Task Learning in Deep Neural Networks
- Enhanced LSTM for Natural Language Inference
- Recurrent Neural Network for Text Classification with Multi-Task Learning
- SQuAD: 100,000+ Questions for Machine Comprehension of Text
- A Survey on Multi-Task Learning
- Domain Separation Networks
- GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
- A large annotated corpus for learning natural language inference
- Multi-Task Deep Neural Networks for Natural Language Understanding
- Multi-Task Cross-Lingual Sequence Tagging from Scratch
- Adversarial Multi-task Learning for Text Classification
- Trace Norm Regularised Deep Multi-Task Learning
- A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks
- Constituency Parsing with a Self-Attentive Encoder