Label Smoothing in Disguise? Gated Knowledge Distillation for Low-Resource Bangla Generation

Dipto Sumit, Zobeida Zaman Chowdhury, Syeda Fabiayee Jahan +1

Gated knowledge distillation (KD) weights teacher signals per token or example, aiming to trust an imperfect teacher only where it seems reliable. We test two gates: EWAD (entropy-…