343 citations · 1.7k across the 69 of their papers we have counts for
5 papers · 1 filter
A Mutual Information Maximization Perspective of Language Representation Learning
Lingpeng Kong, Cyprien de Masson d'Autume, Wang Ling +3
We show state-of-the-art word representation learning methods maximize an objective function that is a lower bound on the mutual information between different parts of a word seque…
Better Document-Level Machine Translation with Bayes' Rule
Lei Yu, Laurent Sartran, Wojciech Stokowiec +4
We show that Bayes' rule provides an effective mechanism for creating document translation models that can be learned from only parallel sentences and monolingual documents---a com…
Episodic Memory in Lifelong Language Learning
Cyprien de Masson d'Autume, Sebastian Ruder, Lingpeng Kong +1
We introduce a lifelong language learning setup where a model needs to learn from a stream of text examples without any dataset identifier. We propose an episodic memory model that…
Learning and Evaluating General Linguistic Intelligence
Dani Yogatama, Cyprien de Masson d'Autume, Jerome Connor +8
We define general linguistic intelligence as the ability to reuse previously acquired knowledge about a language's lexicon, syntax, semantics, and pragmatic conventions to adapt to…
Variational Smoothing in Recurrent Neural Network Language Models
Lingpeng Kong, Gabor Melis, Wang Ling +2
We present a new theoretical perspective of data noising in recurrent neural network language models (Xie et al., 2017). We show that each variant of data noising is an instance of…