4 papers
Efficiently Training A Flat Neural Network Before It has been Quantizated
Peng Xia, Junbiao Pang, Tianyang Cai
Post-training quantization (PTQ) for vision transformers (ViTs) has garnered significant attention due to its efficiency in compressing models. However, existing methods typically…
MEC-Quant: Maximum Entropy Coding for Extremely Low Bit Quantization-Aware Training
Junbiao Pang, Tianyang Cai, Baochang Zhang
Quantization-Aware Training (QAT) has driven much attention to produce efficient neural networks. Current QAT still obtains inferior performances compared with the Full Precision (…
Stabilizing Quantization-Aware Training by Implicit-Regularization on Hessian Matrix
Junbiao Pang, Tianyang Cai
Quantization-Aware Training (QAT) is one of the prevailing neural network compression solutions. However, its stability has been challenged for yielding deteriorating performances…
In-Distribution Consistency Regularization Improves the Generalization of Quantization-Aware Training
Junbiao Pang, Tianyang Cai, Baochang Zhang +1
Although existing Quantization-Aware Training (QAT) methods intensively depend on knowledge distillation to guarantee performance, QAT still suffers from severe performance drop. T…