Chain-based Distillation for Effective Initialization of Variable-Sized Small Language Models

Boyu Shi, YiCheng Jiang, Chang Liu +3

Large language models (LLMs) achieve strong performance but remain costly to deploy in resource-constrained settings. Training small language models (SLMs) from scratch is computat…