2 papers
cs.CL2024
Sentence-Level or Token-Level? A Comprehensive Study on Knowledge Distillation
Jingxuan Wei, Linzhuang Sun, Yichong Leng +3
Knowledge distillation, transferring knowledge from a teacher model to a student model, has emerged as a powerful technique in neural machine translation for compressing models or…
cs.CL2023
Unraveling Key Factors of Knowledge Distillation
Jingxuan Wei, Linzhuang Sun, Xu Tan +2
Knowledge distillation, a technique for model compression and performance enhancement, has gained significant traction in Neural Machine Translation (NMT). However, existing resear…