FASQ: Flexible Accelerated Subspace Quantization for Calibration-Free LLM Compression

Ye Qiao, Yian Wang, Zhiheng Chen +2

Compressing large language models (LLMs) for deployment on commodity GPUs remains challenging: conventional scalar quantization is limited to fixed bit-widths (e.g., 8/4/3-bit), of…