FlexQ: Efficient Post-training INT6 Quantization for LLM Serving via Algorithm-System Co-Design

Hao Zhang, Aining Jia, Weifeng Bu +4

Large Language Models (LLMs) demonstrate exceptional performance but entail significant memory and computational costs, restricting their practical deployment. While existing INT4/…