553 citations · 882 across the 39 of their papers we have counts for
11 papers · 1 filter
First-Order Error Matters: Accurate Compensation for Quantized Large Language Models
Xingyu Zheng, Haotong Qin, Yuye Li +5
Post-training quantization (PTQ) offers an efficient approach to compressing large language models (LLMs), significantly reducing memory access and computational costs. Existing co…
SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Models
Wei Huang, Haotong Qin, Yangdong Liu +7
Post-training quantization (PTQ) is an effective technique for compressing large language models (LLMs). However, while uniform-precision quantization is computationally efficient,…
An empirical study of LLaMA3 quantization: from LLMs to MLLMs
Wei Huang, Xingyu Zheng, Xudong Ma +7
The LLaMA family, a collection of foundation language models ranging from 7B to 65B parameters, has become one of the most powerful open-source large language models (LLMs) and the…
DB-LLM: Accurate Dual-Binarization for Efficient LLMs
Hong Chen, Chengtao Lv, Liang Ding +8
Large language models (LLMs) have significantly advanced the field of natural language processing, while the expensive memory and computation consumption impede their practical dep…
Accurate LoRA-Finetuning Quantization of LLMs via Information Retention
Haotong Qin, Xudong Ma, Xingyu Zheng +6
The LoRA-finetuning quantization of LLMs has been extensively studied to obtain accurate yet compact LLMs for deployment on resource-constrained hardware. However, existing methods…
BiLLM: Pushing the Limit of Post-Training Quantization for LLMs
Wei Huang, Yangdong Liu, Haotong Qin +5
Pretrained large language models (LLMs) exhibit exceptional general language processing capabilities but come with significant demands on memory and computational resources. As a p…
RdimKD: Generic Distillation Paradigm by Dimensionality Reduction
Yi Guo, Yiqian He, Xiaoyang Li +4
Knowledge Distillation (KD) emerges as one of the most promising compression technologies to run advanced deep neural networks on resource-limited devices. In order to train a smal…
On-Chip Hardware-Aware Quantization for Mixed Precision Neural Networks
Wei Huang, Haotong Qin, Yangdong Liu +4
Low-bit quantization emerges as one of the most promising compression approaches for deploying deep neural networks on edge devices. Mixed-precision quantization leverages a mixtur…
RobustMQ: Benchmarking Robustness of Quantized Models
Yisong Xiao, Aishan Liu, Tianyuan Zhang +3
Quantization has emerged as an essential technique for deploying deep neural networks (DNNs) on devices with limited resources. However, quantized models exhibit vulnerabilities wh…
Benchmarking the Robustness of Quantized Models
Yisong Xiao, Tianyuan Zhang, Shunchang Liu +1
Quantization has emerged as an essential technique for deploying deep neural networks (DNNs) on devices with limited resources. However, quantized models exhibit vulnerabilities wh…
Exploring Social Posterior Collapse in Variational Autoencoder for Interaction Modeling
Chen Tang, Wei Zhan, Masayoshi Tomizuka
Multi-agent behavior modeling and trajectory forecasting are crucial for the safe navigation of autonomous agents in interactive scenarios. Variational Autoencoder (VAE) has been w…