44 citations · 127 across the 13 of their papers we have counts for
22 papers
Understanding and Improving Lexical Choice in Non-Autoregressive Translation
Liang Ding, Longyue Wang, Xuebo Liu +3
Knowledge distillation (KD) is essential for training non-autoregressive translation (NAT) models by reducing the complexity of the raw data with an autoregressive teacher model. I…
Understanding and Improving Encoder Layer Fusion in Sequence-to-Sequence Learning
Xuebo Liu, Longyue Wang, Derek F. Wong +3
Encoder layer fusion (EncoderFusion) is a technique to fuse all the encoder layers (instead of the uppermost layer) for sequence-to-sequence (Seq2Seq) models, which has proven effe…
Context-Aware Cross-Attention for Non-Autoregressive Translation
Liang Ding, Longyue Wang, Di Wu +2
Non-autoregressive translation (NAT) significantly accelerates the inference process by predicting the entire target sequence. However, due to the lack of target dependency modelli…
On the Sub-Layer Functionalities of Transformer Decoder
Yilin Yang, Longyue Wang, Shuming Shi +3
There have been significant efforts to interpret the encoder of Transformer-based encoder-decoder architectures for neural machine translation (NMT); meanwhile, the decoder remains…
On the Sparsity of Neural Machine Translation Models
Yong Wang, Longyue Wang, Victor O. K. Li +1
Modern neural machine translation (NMT) models employ a large number of parameters, which leads to serious over-parameterization and typically causes the underutilization of comput…
How Does Selective Mechanism Improve Self-Attention Networks?
Xinwei Geng, Longyue Wang, Xing Wang +3
Self-attention networks (SANs) with selective mechanism has produced substantial improvements in various NLP tasks by concentrating on a subset of input words. However, the underly…