132 citations · 161 across the 15 of their papers we have counts for
3 papers · 1 filter
ALP-KD: Attention-Based Layer Projection for Knowledge Distillation
Peyman Passban, Yimeng Wu, Mehdi Rezagholizadeh +1
Knowledge distillation is considered as a training and compression strategy in which two neural networks, namely a teacher and a student, are coupled together during training. The…
Revisiting Robust Neural Machine Translation: A Transformer Case Study
Peyman Passban, Puneeth S. M. Saladi, Qun Liu
Transformers (Vaswani et al., 2017) have brought a remarkable improvement in the performance of neural machine translation (NMT) systems but they could be surprisingly vulnerable t…
Why Skip If You Can Combine: A Simple Knowledge Distillation Technique for Intermediate Layers
Yimeng Wu, Peyman Passban, Mehdi Rezagholizade +1
With the growth of computing power neural machine translation (NMT) models also grow accordingly and become better. However, they also become harder to deploy on edge devices due t…