FlashQuant: Sparse-Dense Fusion for Memory-Efficient Outlier-Aware LLM Inference

Junqing Lin, Jingwei Sun, Zhengding Hu +1

Low-bit quantization reduces the memory footprint and computational cost of large language model (LLM) inference. However, high-magnitude outlier weights can induce substantial qua…