7 papers · 1 filter
SOAR: Scale Optimization for Accurate Reconstruction in NVFP4 Quantization
Chengzhu Bao, Xianglong Yan, Zhiteng Li +3
NVFP4 has recently emerged as an efficient 4-bit microscaling format for large language models (LLMs), offering superior numerical fidelity with native hardware support. However, e…
DQuant: Accurate Low-bit Post-Training Weight Quantization for LLMs
Xianglong Yan, ChengZhu Bao, Zhiteng Li +5
Large language models (LLMs) deliver strong performance, but their high compute and memory costs make deployment difficult in resource-constrained scenarios. Weight-only post-train…
PT-LLM: Post-Training Ternarization for Large Language Models
Xianglong Yan, Chengzhu Bao, Zhiteng Li +6
Large Language Models (LLMs) have shown impressive capabilities across diverse tasks, but their large memory and compute demands hinder deployment. Ternarization has gained attenti…
Quant-dLLM: Post-Training Extreme Low-Bit Quantization for Diffusion Large Language Models
Tianao Zhang, Zhiteng Li, Xianglong Yan +3
Diffusion large language models (dLLMs), which offer bidirectional context and flexible masked-denoising generation, are emerging as a compelling alternative to autoregressive (AR)…
Low-bit Model Quantization for Deep Neural Networks: A Survey
Kai Liu, Qian Zheng, Kaiwen Tao +9
With unprecedented rapid development, deep neural networks (DNNs) have deeply influenced almost all fields. However, their heavy computation costs and model sizes are usually unacc…
ReCalKV: Low-Rank KV Cache Compression via Head Reordering and Offline Calibration
Xianglong Yan, Zhiteng Li, Tianao Zhang +4
Large language models (LLMs) have demonstrated remarkable performance, but their long-context reasoning remains constrained by the excessive memory required for the Key-Value (KV)…
Progressive Binarization with Semi-Structured Pruning for LLMs
Xianglong Yan, Tianao Zhang, Zhiteng Li +2
Large language models (LLMs) have achieved remarkable progress in natural language processing, but their high computational and memory costs hinder deployment on resource-constrain…