AMS-QUANT: Adaptive Mantissa Sharing for Floating-point Quantization

Mengtao Lv, Ruiqi Zhu, Xinyu Wang +1

Large language models (LLMs) have demonstrated remarkable capabilities in various kinds of tasks, while the billion or even trillion parameters bring storage and efficiency bottlen…