Learning Generic Sentence Representations Using Convolutional Neural Networks

Abstract

We propose a new encoder-decoder approach to learn distributed sentence representations that are applicable to multiple purposes. The model is learned by using a convolutional neural network as an encoder to map an input sentence into a continuous vector, and using a long short-term memory recurrent neural network as a decoder. Several tasks are considered, including sentence reconstruction and future sentence prediction. Further, a hierarchical encoder-decoder model is proposed to encode a sentence to predict multiple future sentences. By training our models on a large collection of novels, we obtain a highly generic convolutional sentence encoder that performs well in practice. Experimental results on several benchmark datasets, and across a broad range of applications, demonstrate the superiority of the proposed model over competing methods.

arXiv commentAccepted by EMNLP 2017

References in corpus (13)

• Sequence to Sequence Learning with Neural Networks
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• A Structured Self-attentive Sentence Embedding
• Theano: new features and speed improvements
• Convolutional Neural Network Architectures for Matching Natural Language Sentences
• Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales
• Unsupervised Learning of Sentence Embeddings using Compositional n-Gram Features
• A Convolutional Neural Network for Modelling Sentences
• Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books
• A Hierarchical Neural Autoencoder for Paragraphs and Documents
• A Convolutional Encoder Model for Neural Machine Translation
• Encoding Source Language with Convolutional Neural Network for Machine Translation
• Character-level Deep Conflation for Business Data Analytics

Cited by in corpus (7)

• An efficient framework for learning sentence representations
• Adversarial Feature Matching for Text Generation
• Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning
• JointGAN: Multi-Domain Joint Distribution Learning with Generative Adversarial Nets
• Learning Context-Sensitive Convolutional Filters for Text Processing
• Character-level Deep Conflation for Business Data Analytics
• Latte-Mix: Measuring Sentence Semantic Similarity with Latent Categorical Mixtures