AutoBSS: An Efficient Algorithm for Block Stacking Style Search

Abstract

Neural network architecture design mostly focuses on the new convolutional operator or special topological structure of network block, little attention is drawn to the configuration of stacking each block, called Block Stacking Style (BSS). Recent studies show that BSS may also have an unneglectable impact on networks, thus we design an efficient algorithm to search it automatically. The proposed method, AutoBSS, is a novel AutoML algorithm based on Bayesian optimization by iteratively refining and clustering Block Stacking Style Code (BSSC), which can find optimal BSS in a few trials without biased evaluation. On ImageNet classification task, ResNet50/MobileNetV2/EfficientNet-B0 with our searched BSS achieve 79.29%/74.5%/77.79%, which outperform the original baselines by a large margin. More importantly, experimental results on model compression, object detection and instance segmentation show the strong generalizability of the proposed AutoBSS, and further verify the unneglectable impact of BSS on neural networks.

arXiv commentNeurIPS 2020

References in corpus (12)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Neural Architecture Search with Reinforcement Learning
• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
• A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning
• DARTS: Differentiable Architecture Search
• Large-Scale Evolution of Image Classifiers
• Random Search and Reproducibility for Neural Architecture Search
• Evaluating the Search Phase of Neural Architecture Search
• FasterSeg: Searching for Faster Real-time Semantic Segmentation
• NAS evaluation is frustratingly hard
• Good Subnetworks Provably Exist: Pruning via Greedy Forward Selection
• BETANAS: BalancEd TrAining and selective drop for Neural Architecture Search