CCQ: Convolutional Code for Extreme Low-bit Quantization in LLMs

Zhaojing Zhou, Xunchao Li, Minghao Li +8

The rapid scaling of Large Language Models (LLMs) elevates inference costs and compounds substantial deployment barriers. While quantization to 8 or 4 bits mitigates this, sub-3-bi…