2.1k citations · 4.2k across the 21 of their papers we have counts for
4 papers · 1 filter
Rethinking embedding coupling in pre-trained language models
Hyung Won Chung, Thibault Févry, Henry Tsai +2
We re-evaluate the standard practice of sharing weights between input and output embeddings in state-of-the-art pre-trained language models. We show that decoupled embeddings provi…
Improving Multilingual Models with Language-Clustered Vocabularies
Hyung Won Chung, Dan Garrette, Kiat Chuan Tan +1
State-of-the-art multilingual models depend on vocabularies that cover all of the languages the model will expect to see at inference time, but the standard methods for generating…
Learning to Evaluate Translation Beyond English: BLEURT Submissions to the WMT Metrics 2020 Shared Task
Thibault Sellam, Amy Pu, Hyung Won Chung +5
The quality of machine translation systems has dramatically improved over the last decade, and as a result, evaluation has become an increasingly challenging problem. This paper de…
Finding Fast Transformers: One-Shot Neural Architecture Search by Component Composition
Henry Tsai, Jayden Ooi, Chun-Sung Ferng +2
Transformer-based models have achieved stateof-the-art results in many tasks in natural language processing. However, such models are usually slow at inference time, making deploym…