128 citations · 307 across the 13 of their papers we have counts for
Showing 2021Show all
3 papers · 1 filter
cs.CL2021
Language Scaling for Universal Suggested Replies Model
Qianlan Ying, Payal Bajaj, Budhaditya Deb +7
We consider the problem of scaling automated suggested replies for Outlook email system to multiple languages. Faced with increased compute requirements and low resources for langu…
cs.CL2021★ 24 cited
XLM-E: Cross-lingual Language Model Pre-training via ELECTRA
Zewen Chi, Shaohan Huang, Li Dong +8
In this paper, we introduce ELECTRA-style tasks to cross-lingual language model pre-training. Specifically, we present two pre-training tasks, namely multilingual replaced token de…
cs.CL2021★ 128 cited
COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining
Yu Meng, Chenyan Xiong, Payal Bajaj +4
We present a self-supervised learning framework, COCO-LM, that pretrains Language Models by COrrecting and COntrasting corrupted text sequences. Following ELECTRA-style pretraining…