4 papers
Self-Supervised Learning for Sparse Matrix Reordering
Ziwei Li, Tao Yuan, Fangfang Liu +3
Rearranging the rows or columns of a sparse matrix using an appropriate ordering can significantly reduce fill-ins, i.e., new nonzeros introduced during matrix factorization, decre…
TileQ: Efficient Low-Rank Quantization of Mixture-of-Experts with 2D Tiling
Hongyaoxing Gu, Xinzhe Chen, Lijuan Hu +1
Mixture-of-Experts (MoE) models achieve remarkable performance by sparsely activating specialized experts, yet their massive parameters in experts pose significant challenges for d…
LoPRo: Enhancing Low-Rank Quantization via Permuted Block-Wise Rotation
Hongyaoxing Gu, Lijuan Hu, Liye Yu +2
Post-training quantization (PTQ) enables effective model compression while preserving relatively high accuracy. Current weight-only PTQ methods primarily focus on the challenging s…
FLRQ: Faster LLM Quantization with Flexible Low-Rank Matrix Sketching
Hongyaoxing Gul, Lijuan Hu, Shuzi Niu +1
Traditional post-training quantization (PTQ) is considered an effective approach to reduce model size and accelerate inference of large-scale language models (LLMs). However, exist…