Pushing the Limits of Large Language Model Quantization via the Linearity Theorem

Vladimir Malinovskii, Andrei Panferov, Ivan Ilin +3

Quantizing large language models has become a standard way to reduce their memory and computational costs. Typically, existing methods focus on breaking down the problem into indiv…