From the 1 of 18 linked papers with an AI index.
12 papers · 1 filter
SandwichQuant: Which Parameters Matter Before and After Quantization?
Peng Xia, Junbiao Pang
Quantization correction methods usually optimize weights, quantization parameters, or reconstruction objectives, while the underlying parameter subspaces responsible for effective…
Low-Dimensional High-Leverage Subspace Optimization: Beyond Full-Parameter Coupled Training for Neural Network Quantization
Peng Xia, Junbiao Pang, Zheng Huang
Low-bit quantization suffers severe accuracy degradation on compact networks, rooted in the dominant full-parameter coupled training paradigm that ignores parameter subspace hetero…
Efficient Tuning Before Low-Bit Post-Training Quantization for Stochastic Gradient Descent-optimized Models
Peng Xia, Junbiao Pang, Muhammad Ayub Sabir
The paper introduces Efficient Tuning Before Quantization (ETBQ), a lightweight pre‑conditioning step that adjusts a full‑precision model using perturbations from quantization erro…
Efficiently Training A Flat Neural Network Before It has been Quantizated
Peng Xia, Junbiao Pang, Tianyang Cai
Post-training quantization (PTQ) for vision transformers (ViTs) has garnered significant attention due to its efficiency in compressing models. However, existing methods typically…
Adaptively Sampling-Reusing-Mixing Decomposed Gradients to Speed Up Sharpness Aware Minimization
Jiaxin Deng, Junbiao Pang
Sharpness-Aware Minimization (SAM) improves model generalization but doubles the computational cost of Stochastic Gradient Descent (SGD) by requiring twice the gradient calculation…
MEC-Quant: Maximum Entropy Coding for Extremely Low Bit Quantization-Aware Training
Junbiao Pang, Tianyang Cai, Baochang Zhang
Quantization-Aware Training (QAT) has driven much attention to produce efficient neural networks. Current QAT still obtains inferior performances compared with the Full Precision (…