238 citations · 694 across the 33 of their papers we have counts for
8 papers · 1 filter
Data-Free Neural Architecture Search via Recursive Label Calibration
Zechun Liu, Zhiqiang Shen, Yun Long +3
This paper aims to explore the feasibility of neural architecture search (NAS) given only a pre-trained model without using any original training data. This is an important circums…
Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimation
Zechun Liu, Kwang-Ting Cheng, Dong Huang +2
The nonuniform quantization strategy for compressing neural networks usually achieves better performance than its counterpart, i.e., uniform strategy, due to its superior represent…
Sliced Recursive Transformer
Zhiqiang Shen, Zechun Liu, Eric Xing
We present a neat yet effective recursive operation on vision transformers that can improve parameter utilization without involving additional parameters. This is achieved by shari…
How Do Adam and Training Strategies Help BNNs Optimization?
Zechun Liu, Zhiqiang Shen, Shichao Li +3
The best performing Binary Neural Networks (BNNs) are usually attained using Adam optimization and its multi-step training variants. However, to the best of our knowledge, few stud…
"BNN - BN = ?": Training Binary Neural Networks without Batch Normalization
Tianlong Chen, Zhenyu Zhang, Xu Ouyang +3
Batch normalization (BN) is a key facilitator and considered essential for state-of-the-art binary neural networks (BNN). However, the BN layer is costly to calculate and is typica…
Is Label Smoothing Truly Incompatible with Knowledge Distillation: An Empirical Study
Zhiqiang Shen, Zechun Liu, Dejia Xu +3
This work aims to empirically clarify a recently discovered perspective that label smoothing is incompatible with knowledge distillation. We begin by introducing the motivation beh…
Partial Is Better Than All: Revisiting Fine-tuning Strategy for Few-shot Learning
Zhiqiang Shen, Zechun Liu, Jie Qin +2
The goal of few-shot learning is to learn a classifier that can recognize unseen classes from limited support data with labels. A common practice for this task is to train a model…
S2-BNN: Bridging the Gap Between Self-Supervised Real and 1-bit Neural Networks via Guided Distribution Calibration
Zhiqiang Shen, Zechun Liu, Jie Qin +3
Previous studies dominantly target at self-supervised learning on real-valued networks and have achieved many promising results. However, on the more challenging binary neural netw…