Iterative Layer-wise Distillation for Efficient Compression of Large Language Models

Grigory Kovalev, Mikhail Tikhomirov

This work investigates distillation methods for large language models (LLMs) with the goal of developing compact models that preserve high performance. Several existing approaches…