Accurate Block Quantization in LLMs with Outliers

Nikita Trukhanov, Ilya Soloveychik

The demand for inference on extremely large scale LLMs has seen enormous growth in the recent months. It made evident the colossal shortage of dedicated hardware capable of efficie…