4 citations · 5 across the 25 of their papers we have counts for
29 papers
Fine-Grained Units, Flat Decisions: Hierarchical Micro-Expert Pruning for Mixture-of-Experts Models
Zukang Xu, Xinpei Gao, Xing Hu +2
Micro-expert pruning reduces the capacity of mixture-of-experts (MoE) models by removing coupled channels within routed experts. However, finer pruning granularity alone does not g…
LoopQuant: Accurate and Efficient Post-Training Quantization for Looped Transformers
Zhixiong Zhao, Zukang Xu, Xing Hu +3
Looped Language Models (LoopLMs) improve parameter efficiency by recursively reusing shared Transformer modules, yet their repeated computation over evolving recurrent states poses…
All for 1-Bit: Towards Genuine 1-Bit Post-Training Quantization for LLMs
Zhixiong Zhao, Zukang Xu, Guangyu Sun +2
Large language models (LLMs) have achieved remarkable progress, yet their massive storage and memory-bandwidth demands still hinder efficient deployment. Weight binarization is a p…
ACE: Adaptive Calibration-Free Expert Skipping for MoE-based LLMs
Zukang Xu, Zhixiong Zhao, Xing Hu +5
Mixture-of-Experts (MoE) architectures provide an efficient paradigm for scaling large language models (LLMs), yet fixed top-k routing activates the same number of expert slots for…
MGVQ: Synergizing Multi-dimensional Sensitivity-Aware and Gradient-Hessian Fusion for Vector Quantization
Zhong Wang, Zukang Xu, Xing Hu +1
Vision-Language Models (VLMs) achieve outstanding performance, yet their huge model size severely hinders deployment on edge devices with limited resources. As an efficient model c…
TORQ: Two-Level Orthogonal Rotation for MXFP4 Quantization
Zukang Xu, Xing Hu, Dawei Yang
As Large Language Models (LLMs) advance toward practical deployment, the Microscaling FP4 (MXFP4) format has emerged as a cornerstone for next-generation low-bit inference, owing t…
CAR-SAM: Cross-Attention Reconstruction for Post-Training Quantization of the Segment Anything Model
Houji Wen, Jiangyong Yu, Jun Li +1
Segment Anything Models (SAMs) are extensively used in computer vision for universal image segmentation, but deploying them on resource-constrained devices is challenging due to th…
R3-VAE: Reference Vector-Guided Rating Residual Quantization VAE for Generative Recommendation
Qiang Wan, Ze Yang, Dawei Yang +8
Generative Recommendation (GR) has gained traction for its merits of superior performance and cold-start capability. As the vital role in GR, Semantic Identifiers (SIDs) represent…
BWLA: Breaking the Barrier of W1AX Post-Training Quantization for LLMs
Zhixiong Zhao, Zukang Xu, Dawei Yang
Large language models (LLMs) have driven major progress in NLP, yet their substantial memory and compute demands still hinder practical deployment. Binarization can compress weight…
MoBiE: Efficient Inference of Mixture of Binary Experts under Post-Training Quantization
Zhixiong Zhao, Zukang Xu, Zhixuan Chen +1
Mixture-of-Experts (MoE) based large language models (LLMs) offer strong performance but suffer from high memory and computation costs. Weight binarization provides extreme efficie…
KBVQ-MoE: KLT-guided SVD with Bias-Corrected Vector Quantization for MoE Large Language Models
Zukang Xu, Zhixiong Zhao, Xing Hu +2
Mixture of Experts (MoE) models have achieved great success by significantly improving performance while maintaining computational efficiency through sparse expert activation. Howe…
SAES-SVD: Self-Adaptive Suppression of Accumulated and Local Errors for SVD-based LLM Compression
Xing Hu, Dawei Yang, Yuan Cheng +2
The rapid growth in the parameter scale of large language models (LLMs) has created a high demand for efficient compression techniques. As a hardware-agnostic and highly compatible…
NLI:Non-uniform Linear Interpolation Approximation of Nonlinear Operations for Efficient LLMs Inference
Jiangyong Yu, Xiaomeng Han, Xing Hu +3
Large Language Models (LLMs) have demonstrated remarkable performance across a wide range of tasks, but their deployment is often constrained by substantial memory footprints and c…
OTARo: Once Tuning for All Precisions toward Robust On-Device LLMs
Shaoyuan Chen, Zhixuan Chen, Dawei Yang +2
Large Language Models (LLMs) fine-tuning techniques not only improve the adaptability to diverse downstream tasks, but also mitigate adverse effects of model quantization. Despite…
VAEVQ: Enhancing Discrete Visual Tokenization through Variational Modeling
Sicheng Yang, Xing Hu, Qiang Wu +1
Vector quantization (VQ) transforms continuous image features into discrete representations, providing compressed, tokenized inputs for generative models. However, VQ-based framewo…
RSAVQ: Riemannian Sensitivity-Aware Vector Quantization for Large Language Models
Zukang Xu, Xing Hu, Qiang Wu +1
Large language models (LLMs) have demonstrated remarkable performance across a wide range of natural language processing tasks. However, their exponentially increasing parameters p…
DLLMQuant: Quantizing Diffusion-based Large Language Models
Chen Xu, Dawei Yang
Diffusion-based large language models (DLLMs) have shown promise for non-autoregressive text generation, but their deployment is constrained by large model sizes and heavy computat…
EA-ViT: Efficient Adaptation for Elastic Vision Transformer
Chen Zhu, Wangbo Zhao, Huiwen Zhang +9
Vision Transformers (ViTs) have emerged as a foundational model in computer vision, excelling in generalization and adaptation to downstream tasks. However, deploying ViTs to suppo…
PCDVQ: Enhancing Vector Quantization for Large Language Models via Polar Coordinate Decoupling
Yuxuan Yue, Zukang Xu, Zhihang Yuan +3
Large Language Models (LLMs) face significant challenges in edge deployment due to their massive parameter scale. Vector Quantization (VQ), a clustering-based quantization method,…
MoEQuant: Enhancing Quantization for Mixture-of-Experts Large Language Models via Expert-Balanced Sampling and Affinity Guidance
Xing Hu, Zhixuan Chen, Dawei Yang +5
Mixture-of-Experts (MoE) large language models (LLMs), which leverage dynamic routing and sparse activation to enhance efficiency and scalability, have achieved higher performance…
RWKVQuant: Quantizing the RWKV Family with Proxy Guided Hybrid of Scalar and Vector Quantization
Chen Xu, Yuxuan Yue, Zukang Xu +6
RWKV is a modern RNN architecture with comparable performance to Transformer, but still faces challenges when deployed to resource-constrained devices. Post Training Quantization (…
GSQ-Tuning: Group-Shared Exponents Integer in Fully Quantized Training for LLMs On-Device Fine-tuning
Sifan Zhou, Shuo Wang, Zhihang Yuan +3
Large Language Models (LLMs) fine-tuning technologies have achieved remarkable results. However, traditional LLM fine-tuning approaches face significant challenges: they require la…
MQuant: Unleashing the Inference Potential of Multimodal Large Language Models via Full Static Quantization
JiangYong Yu, Sifan Zhou, Dawei Yang +7
Multimodal large language models (MLLMs) have garnered widespread attention due to their ability to understand multimodal input. However, their large parameter sizes and substantia…
OstQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fitting
Xing Hu, Yuan Cheng, Dawei Yang +6
Post-training quantization (PTQ) has emerged as a widely adopted technique for compressing and accelerating Large Language Models (LLMs). The major challenge in LLM quantization is…