49 citations · 89 across the 4 of their papers we have counts for
Showing 2022Show all
2 papers · 1 filter
cs.LG2022★ 37 cited
Mega: Moving Average Equipped Gated Attention
Xuezhe Ma, Chunting Zhou, Xiang Kong +5
The design choices in the Transformer attention mechanism, including weak inductive bias and quadratic computational complexity, have limited its application for modeling long sequ…
cs.CL2022★ 1 cited
Multilingual Neural Machine Translation with Deep Encoder and Multiple Shallow Decoders
Xiang Kong, Adithya Renduchintala, James Cross +3
Recent work in multilingual translation advances translation quality surpassing bilingual baselines using deep transformer models with increased capacity. However, the extra latenc…