44 citations · 154 across the 14 of their papers we have counts for
23 papers · 1 filter
Macaw-LLM: Multi-Modal Language Modeling with Image, Audio, Video, and Text Integration
Chenyang Lyu, Minghao Wu, Longyue Wang +5
Although instruction-tuned large language models (LLMs) have exhibited remarkable capabilities across various NLP tasks, their effectiveness on other data modalities beyond text ha…
Understanding and Improving Lexical Choice in Non-Autoregressive Translation
Liang Ding, Longyue Wang, Xuebo Liu +3
Knowledge distillation (KD) is essential for training non-autoregressive translation (NAT) models by reducing the complexity of the raw data with an autoregressive teacher model. I…
Understanding and Improving Encoder Layer Fusion in Sequence-to-Sequence Learning
Xuebo Liu, Longyue Wang, Derek F. Wong +3
Encoder layer fusion (EncoderFusion) is a technique to fuse all the encoder layers (instead of the uppermost layer) for sequence-to-sequence (Seq2Seq) models, which has proven effe…
Context-Aware Cross-Attention for Non-Autoregressive Translation
Liang Ding, Longyue Wang, Di Wu +2
Non-autoregressive translation (NAT) significantly accelerates the inference process by predicting the entire target sequence. However, due to the lack of target dependency modelli…
On the Sub-Layer Functionalities of Transformer Decoder
Yilin Yang, Longyue Wang, Shuming Shi +3
There have been significant efforts to interpret the encoder of Transformer-based encoder-decoder architectures for neural machine translation (NMT); meanwhile, the decoder remains…
On the Sparsity of Neural Machine Translation Models
Yong Wang, Longyue Wang, Victor O. K. Li +1
Modern neural machine translation (NMT) models employ a large number of parameters, which leads to serious over-parameterization and typically causes the underutilization of comput…