Learning Robust, Transferable Sentence Representations for Text Classification

Abstract

Despite deep recurrent neural networks (RNNs) demonstrate strong performance in text classification, training RNN models are often expensive and requires an extensive collection of annotated data which may not be available. To overcome the data limitation issue, existing approaches leverage either pre-trained word embedding or sentence representation to lift the burden of training RNNs from scratch. In this paper, we show that jointly learning sentence representations from multiple text classification tasks and combining them with pre-trained word-level and sentence level encoders result in robust sentence representations that are useful for transfer learning. Extensive experiments and analyses using a wide range of transfer and linguistic tasks endorse the effectiveness of our approach.

arXiv commentarXiv admin note: substantial text overlap with arXiv:1804.07911

References in corpus (16)

• Universal Sentence Encoder
• A large annotated corpus for learning natural language inference
• An efficient framework for learning sentence representations
• Learning to Generate Reviews and Discovering Sentiment
• SentEval: An Evaluation Toolkit for Universal Sentence Representations
• Text Classification Improved by Integrating Bidirectional LSTM with Two-dimensional Max Pooling
• What you can cram into a single vector: Probing sentence embeddings for linguistic properties
• Learned in Translation: Contextualized Word Vectors
• Learning Natural Language Inference using Bidirectional LSTM model and Inner-Attention
• Bilateral Multi-Perspective Matching for Natural Language Sentences
• Evaluation of sentence embeddings in downstream and linguistic probing tasks
• Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning
• How Transferable are Neural Networks in NLP Applications?
• Deep Multitask Learning for Semantic Dependency Parsing
• Multi-Task Label Embedding for Text Classification
• Few-Shot Text Classification with Pre-Trained Word Embeddings and a Human in the Loop