Self-Data Distillation for Recovering Quality in Pruned Large Language Models

Vithursan Thangarasa, Ganesh Venkatesh, Mike Lasby +2

Large language models have driven significant progress in natural language processing, but their deployment requires substantial compute and memory resources. As models scale, comp…