Knowledge Distillation of Domain-adapted LLMs for Question-Answering in Telecom

Rishika Sen, Sujoy Roychowdhury, Sumit Soman +2

Knowledge Distillation (KD) is one of the approaches to reduce the size of Large Language Models (LLMs). A LLM with smaller number of model parameters (student) is trained to mimic…