1 citations · 1 across the 9 of their papers we have counts for
9 papers
SURGE: Surrogate Gradient Adaptation in Binary Neural Networks
Haoyu Huang, Boyu Liu, Linlin Yang +6
The training of Binary Neural Networks (BNNs) is fundamentally based on gradient approximation for non-differentiable binarization operations (e.g., sign function). However, prevai…
AMLRIS: Alignment-aware Masked Learning for Referring Image Segmentation
Tongfei Chen, Shuo Yang, Yuguang Yang +7
Referring Image Segmentation (RIS) aims to segment the object in an image uniquely referred to by a natural language expression. However, RIS training often contains hard-to-align…
MEC-Quant: Maximum Entropy Coding for Extremely Low Bit Quantization-Aware Training
Junbiao Pang, Tianyang Cai, Baochang Zhang
Quantization-Aware Training (QAT) has driven much attention to produce efficient neural networks. Current QAT still obtains inferior performances compared with the Full Precision (…
Efficiently Seeking Flat Minima for Better Generalization in Fine-Tuning Large Language Models and Beyond
Jiaxin Deng, Qingcheng Zhu, Junbiao Pang +3
Little research explores the correlation between the expressive ability and generalization ability of the low-rank adaptation (LoRA). Sharpness-Aware Minimization (SAM) improves mo…
Bilateral Sharpness-Aware Minimization for Flatter Minima
Jiaxin Deng, Junbiao Pang, Baochang Zhang +1
Sharpness-Aware Minimization (SAM) enhances generalization by reducing a Max-Sharpness (MaxS). Despite the practical success, we empirically found that the MAxS behind SAM's genera…
Asymptotic Unbiased Sample Sampling to Speed Up Sharpness-Aware Minimization
Jiaxin Deng, Junbiao Pang, Baochang Zhang
Sharpness-Aware Minimization (SAM) has emerged as a promising approach for effectively reducing the generalization error. However, SAM incurs twice the computational cost compared…
A Channel-ensemble Approach: Unbiased and Low-variance Pseudo-labels is Critical for Semi-supervised Classification
Jiaqi Wu, Junbiao Pang, Baochang Zhang +1
Semi-supervised learning (SSL) is a practical challenge in computer vision. Pseudo-label (PL) methods, e.g., FixMatch and FreeMatch, obtain the State Of The Art (SOTA) performances…
Effective Gradient Sample Size via Variation Estimation for Accelerating Sharpness aware Minimization
Jiaxin Deng, Junbiao Pang, Baochang Zhang +1
Sharpness-aware Minimization (SAM) has been proposed recently to improve model generalization ability. However, SAM calculates the gradient twice in each optimization step, thereby…
In-Distribution Consistency Regularization Improves the Generalization of Quantization-Aware Training
Junbiao Pang, Tianyang Cai, Baochang Zhang +1
Although existing Quantization-Aware Training (QAT) methods intensively depend on knowledge distillation to guarantee performance, QAT still suffers from severe performance drop. T…