Multi-Bitwidth Quantization for LLMs Using Additive Codebooks

Liza Babaoglu, Shuangyi Chen, Ashish Khisti

As large language models (LLMs) are increasingly deployed across heterogeneous hardware with varying resource constraints, the ability to adaptively manage the trade-off between pe…