11 papers
BWLA: Breaking the Barrier of W1AX Post-Training Quantization for LLMs
Zhixiong Zhao, Zukang Xu, Dawei Yang
Large language models (LLMs) have driven major progress in NLP, yet their substantial memory and compute demands still hinder practical deployment. Binarization can compress weight…
MoBiE: Efficient Inference of Mixture of Binary Experts under Post-Training Quantization
Zhixiong Zhao, Zukang Xu, Zhixuan Chen +1
Mixture-of-Experts (MoE) based large language models (LLMs) offer strong performance but suffer from high memory and computation costs. Weight binarization provides extreme efficie…
KBVQ-MoE: KLT-guided SVD with Bias-Corrected Vector Quantization for MoE Large Language Models
Zukang Xu, Zhixiong Zhao, Xing Hu +2
Mixture of Experts (MoE) models have achieved great success by significantly improving performance while maintaining computational efficiency through sparse expert activation. Howe…
SAES-SVD: Self-Adaptive Suppression of Accumulated and Local Errors for SVD-based LLM Compression
Xing Hu, Dawei Yang, Yuan Cheng +2
The rapid growth in the parameter scale of large language models (LLMs) has created a high demand for efficient compression techniques. As a hardware-agnostic and highly compatible…
RSAVQ: Riemannian Sensitivity-Aware Vector Quantization for Large Language Models
Zukang Xu, Xing Hu, Qiang Wu +1
Large language models (LLMs) have demonstrated remarkable performance across a wide range of natural language processing tasks. However, their exponentially increasing parameters p…
PCDVQ: Enhancing Vector Quantization for Large Language Models via Polar Coordinate Decoupling
Yuxuan Yue, Zukang Xu, Zhihang Yuan +3
Large Language Models (LLMs) face significant challenges in edge deployment due to their massive parameter scale. Vector Quantization (VQ), a clustering-based quantization method,…