Universality of Layer-Level Entropy-Weighted Quantization Beyond Model Architecture and Size

Alireza Behtash, Marijan Fofonjka, Ethan Baird +4

We present a novel approach to selective model quantization that transcends the limitations of architecture-specific and size-dependent compression methods for Large Language Model…