BCNet: Searching for Network Width with Bilaterally Coupled Network
arXiv:2105.10533
Abstract
Searching for a more compact network width recently serves as an effective way of channel pruning for the deployment of convolutional neural networks (CNNs) under hardware constraints. To fulfill the searching, a one-shot supernet is usually leveraged to efficiently evaluate the performance \wrt~different network widths. However, current methods mainly follow a \textit{unilaterally augmented} (UA) principle for the evaluation of each width, which induces the training unfairness of channels in supernet. In this paper, we introduce a new supernet called Bilaterally Coupled Network (BCNet) to address this issue. In BCNet, each channel is fairly trained and responsible for the same amount of network widths, thus each network width can be evaluated more accurately. Besides, we leverage a stochastic complementary strategy for training the BCNet, and propose a prior initial population sampling method to boost the performance of the evolutionary search. Extensive experiments on benchmark CIFAR-10 and ImageNet datasets indicate that our method can achieve state-of-the-art or competing performance over other baseline methods. Moreover, our method turns out to further boost the performance of NAS models by refining their network widths. For example, with the same FLOPs budget, our obtained EfficientNet-B0 achieves 77.36\% Top-1 accuracy on ImageNet dataset, surpassing the performance of original setting by 0.48\%.
arXiv commentAccepted by CVPR 2021
References in corpus (15)
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- Distilling the Knowledge in a Neural Network
- AMC: AutoML for Model Compression and Acceleration on Mobile Devices
- Rethinking the Value of Network Pruning
- Discrimination-aware Channel Pruning for Deep Neural Networks
- AutoSlim: Towards One-Shot Architecture Search for Channel Numbers
- Channel Gating Neural Networks
- Gate Decorator: Global Filter Pruning Method for Accelerating Deep Convolutional Neural Networks
- SCOP: Scientific Control for Reliable Neural Network Pruning
- Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks
- Model Rubik's Cube: Twisting Resolution, Depth and Width for TinyNets
- ISTA-NAS: Efficient and Consistent Neural Architecture Search by Sparse Coding
- Locally Free Weight Sharing for Network Width Search
- Bringing Giant Neural Networks Down to Earth with Unlabeled Data
- Towards Improving the Consistency, Efficiency, and Flexibility of Differentiable Neural Architecture Search