CRVQ: Channel-Relaxed Vector Quantization for Extreme Compression of LLMs

Yuzhuang Xu, Shiyu Ji, Qingfu Zhu +1

Powerful large language models (LLMs) are increasingly expected to be deployed with lower computational costs, enabling their capabilities on resource-constrained devices. Post-tra…