49 citations · 89 across the 4 of their papers we have counts for
4 papers
Mega: Moving Average Equipped Gated Attention
Xuezhe Ma, Chunting Zhou, Xiang Kong +5
The design choices in the Transformer attention mechanism, including weak inductive bias and quadratic computational complexity, have limited its application for modeling long sequ…
Multilingual Neural Machine Translation with Deep Encoder and Multiple Shallow Decoders
Xiang Kong, Adithya Renduchintala, James Cross +3
Recent work in multilingual translation advances translation quality surpassing bilingual baselines using deep transformer models with increased capacity. However, the extra latenc…
Luna: Linear Unified Nested Attention
Xuezhe Ma, Xiang Kong, Sinong Wang +4
The quadratic computational and memory complexities of the Transformer's attention mechanism have limited its scalability for modeling long sequences. In this paper, we propose Lun…
Decompressing Knowledge Graph Representations for Link Prediction
Xiang Kong, Xianyang Chen, Eduard Hovy
This paper studies the problem of predicting missing relationships between entities in knowledge graphs through learning their representations. Currently, the majority of existing…