Multi-view Sentence Representation Learning
arXiv:1805.07443
Abstract
Multi-view learning can provide self-supervision when different views are available of the same data. The distributional hypothesis provides another form of useful self-supervision from adjacent sentences which are plentiful in large unlabelled corpora. Motivated by the asymmetry in the two hemispheres of the human brain as well as the observation that different learning architectures tend to emphasise different aspects of sentence meaning, we create a unified multi-view sentence representation learning framework, in which, one view encodes the input sentence with a Recurrent Neural Network (RNN), and the other view encodes it with a simple linear model, and the training objective is to maximise the agreement specified by the adjacent context information between two views. We show that, after training, the vectors produced from our multi-view training provide improved representations over the single-view training, and the combination of different views gives further representational improvement and demonstrates solid transferability on standard downstream tasks.
References in corpus (11)
- Adam: A Method for Stochastic Optimization
- Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
- From Frequency to Meaning: Vector Space Models of Semantics
- Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales
- Layer Normalization
- A large annotated corpus for learning natural language inference
- An efficient framework for learning sentence representations
- Learned in Translation: Contextualized Word Vectors
- All-but-the-Top: Simple and Effective Postprocessing for Word Representations
- Discourse-Based Objectives for Fast Unsupervised Sentence Representation Learning
- DisSent: Sentence Representation Learning from Explicit Discourse Relations