244 citations · 511 across the 34 of their papers we have counts for
6 papers · 1 filter
Understanding and Improving Encoder Layer Fusion in Sequence-to-Sequence Learning
Xuebo Liu, Longyue Wang, Derek F. Wong +3
Encoder layer fusion (EncoderFusion) is a technique to fuse all the encoder layers (instead of the uppermost layer) for sequence-to-sequence (Seq2Seq) models, which has proven effe…
Context-Aware Cross-Attention for Non-Autoregressive Translation
Liang Ding, Longyue Wang, Di Wu +2
Non-autoregressive translation (NAT) significantly accelerates the inference process by predicting the entire target sequence. However, due to the lack of target dependency modelli…
On the Sub-Layer Functionalities of Transformer Decoder
Yilin Yang, Longyue Wang, Shuming Shi +3
There have been significant efforts to interpret the encoder of Transformer-based encoder-decoder architectures for neural machine translation (NMT); meanwhile, the decoder remains…
On the Sparsity of Neural Machine Translation Models
Yong Wang, Longyue Wang, Victor O. K. Li +1
Modern neural machine translation (NMT) models employ a large number of parameters, which leads to serious over-parameterization and typically causes the underutilization of comput…
How Does Selective Mechanism Improve Self-Attention Networks?
Xinwei Geng, Longyue Wang, Xing Wang +3
Self-attention networks (SANs) with selective mechanism has produced substantial improvements in various NLP tasks by concentrating on a subset of input words. However, the underly…
Self-Attention with Cross-Lingual Position Representation
Liang Ding, Longyue Wang, Dacheng Tao
Position encoding (PE), an essential part of self-attention networks (SANs), is used to preserve the word order information for natural language processing tasks, generating fixed…