Compression Scaling Laws:Unifying Sparsity and Quantization

Elias Frantar, Utku Evci, Wonpyo Park +2

We investigate how different compression techniques -- such as weight and activation quantization, and weight sparsity -- affect the scaling behavior of large language models (LLMs…