12 citations · 26 across the 4 of their papers we have counts for
Showing 2020Show all
2 papers · 1 filter
cs.CL2020★ 11 cited
Rethinking embedding coupling in pre-trained language models
Hyung Won Chung, Thibault Févry, Henry Tsai +2
We re-evaluate the standard practice of sharing weights between input and output embeddings in state-of-the-art pre-trained language models. We show that decoupled embeddings provi…
cs.LG2020★ 12 cited
Finding Fast Transformers: One-Shot Neural Architecture Search by Component Composition
Henry Tsai, Jayden Ooi, Chun-Sung Ferng +2
Transformer-based models have achieved stateof-the-art results in many tasks in natural language processing. However, such models are usually slow at inference time, making deploym…