CAT-Q: Cost-efficient and Accurate Ternary Quantization for LLMs

Shigeng Wang, Chao Li, Yangyuxuan Kang +2

In this paper, we present CAT-Q, Cost-efficient and Accurate Ternary Quantization, for compressing and accelerating LLMs. Unlike existing state-of-the-art ternary quantization meth…