AfroXLMR-Comet: Multilingual Knowledge Distillation with Attention Matching for Low-Resource languages

Joshua Sakthivel Raju, Sanjay S, Jaskaran Singh Walia +2

Language model compression through knowledge distillation has emerged as a promising approach for deploying large language models in resource-constrained environments. However, exi…