1.8k citations · 5.3k across the 25 of their papers we have counts for
Showing 2019 · cs.CLShow all
2 papers · 2 filters
cs.CL2019★ 81 cited
A Mutual Information Maximization Perspective of Language Representation Learning
Lingpeng Kong, Cyprien de Masson d'Autume, Wang Ling +3
We show state-of-the-art word representation learning methods maximize an objective function that is a lower bound on the mutual information between different parts of a word seque…
cs.CL2019★ 1.8k cited
XLNet: Generalized Autoregressive Pretraining for Language Understanding
Zhilin Yang, Zihang Dai, Yiming Yang +3
With the capability of modeling bidirectional contexts, denoising autoencoding based pretraining like BERT achieves better performance than pretraining approaches based on autoregr…