When Does Knowledge Distillation Hurt? Reliability-Aware Distillation for Low-Resource Language Summarization

Dipto Sumit, Ankan Kumar Roy Srizon, Sadia Khair Rodela +4

Knowledge distillation (KD) is a standard approach for compressing sequence-to-sequence models, but its per-sample effects are rarely examined. On the BanSum Bangla summarization b…