5 citations · 8 across the 5 of their papers we have counts for
10 papers
Warmup-Distill: Bridge the Distribution Mismatch between Teacher and Student before Knowledge Distillation
Zengkui Sun, Yijin Liu, Fandong Meng +3
The widespread deployment of Large Language Models (LLMs) is hindered by the high computational demands, making knowledge distillation (KD) crucial for developing compact smaller o…
Enhancing Cross-Tokenizer Knowledge Distillation with Contextual Dynamical Mapping
Yijie Chen, Yijin Liu, Fandong Meng +3
Knowledge Distillation (KD) has emerged as a prominent technique for model compression. However, conventional KD approaches primarily focus on homogeneous architectures with identi…
Scheduled Sampling Based on Decoding Steps for Neural Machine Translation
Yijin Liu, Fandong Meng, Yufeng Chen +2
Scheduled sampling is widely used to mitigate the exposure bias problem for neural machine translation. Its core motivation is to simulate the inference scene during training by re…
Confidence-Aware Scheduled Sampling for Neural Machine Translation
Yijin Liu, Fandong Meng, Yufeng Chen +2
Scheduled sampling is an effective method to alleviate the exposure bias problem of neural machine translation. It simulates the inference scene by randomly replacing ground-truth…
Prevent the Language Model from being Overconfident in Neural Machine Translation
Mengqi Miao, Fandong Meng, Yijin Liu +2
The Neural Machine Translation (NMT) model is essentially a joint language model conditioned on both the source sentence and partial translation. Therefore, the NMT model naturally…
Bilingual Mutual Information Based Adaptive Training for Neural Machine Translation
Yangyifan Xu, Yijin Liu, Fandong Meng +3
Recently, token-level adaptive training has achieved promising improvement in machine translation, where the cross-entropy loss function is adjusted by assigning different training…