Multi-task Learning for Universal Sentence Embeddings: A Thorough Evaluation using Transfer and Auxiliary Tasks
arXiv:1804.07911
Abstract
Learning distributed sentence representations is one of the key challenges in natural language processing. Previous work demonstrated that a recurrent neural network (RNNs) based sentence encoder trained on a large collection of annotated natural language inference data, is efficient in the transfer learning to facilitate other related tasks. In this paper, we show that joint learning of multiple tasks results in better generalizable sentence representations by conducting extensive experiments and analysis comparing the multi-task and single-task learned sentence encoders. The quantitative analysis using auxiliary tasks show that multi-task learning helps to embed better semantic information in the sentence representations compared to single-task learning. In addition, we compare multi-task sentence encoders with contextualized word representations and show that combining both of them can further boost the performance of transfer learning.
References in corpus (14)
- A Structured Self-attentive Sentence Embedding
- VQA: Visual Question Answering
- A large annotated corpus for learning natural language inference
- An efficient framework for learning sentence representations
- Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks
- Learned in Translation: Contextualized Word Vectors
- Learning Natural Language Inference using Bidirectional LSTM model and Inner-Attention
- Bilateral Multi-Perspective Matching for Natural Language Sentences
- Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning
- How Transferable are Neural Networks in NLP Applications?
- End-to-End Neural Sentence Ordering Using Pointer Network
- Deep Multi-Task Learning with Shared Memory
- Deep Multitask Learning for Semantic Dependency Parsing
- Multi-Task Label Embedding for Text Classification