ENAS4D: Efficient Multi-stage CNN Architecture Search for Dynamic Inference

Abstract

Dynamic inference is a feasible way to reduce the computational cost of convolutional neural network(CNN), which can dynamically adjust the computation for each input sample. One of the ways to achieve dynamic inference is to use multi-stage neural network, which contains a sub-network with prediction layer at each stage. The inference of a input sample can exit from early stage if the prediction of the stage is confident enough. However, design a multi-stage CNN architecture is a non-trivial task. In this paper, we introduce a general framework, ENAS4D, which can efficiently search for optimal multi-stage CNN architecture for dynamic inference in a well-designed search space. Firstly, we propose a method to construct the search space with multi-stage convolution. The search space include different numbers of layers, different kernel sizes and different numbers of channels for each stage and the resolution of input samples. Then, we train a once-for-all network that supports to sample diverse multi-stage CNN architecture. A specialized multi-stage network can be obtained from the once-for-all network without additional training. Finally, we devise a method to efficiently search for the optimal multi-stage network that trades the accuracy off the computational cost taking the advantage of once-for-all network. The experiments on the ImageNet classification task demonstrate that the multi-stage CNNs searched by ENAS4D consistently outperform the state-of-the-art method for dyanmic inference. In particular, the network achieves 74.4% ImageNet top-1 accuracy under 185M average MACs.

arXiv comment6 figures

References in corpus (7)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Neural Architecture Search with Reinforcement Learning
• DARTS: Differentiable Architecture Search
• Deep Learning with Limited Numerical Precision
• More is Less: A More Complicated Network with Less Inference Complexity
• Dynamically Sacrificing Accuracy for Reduced Computation: Cascaded Inference Based on Softmax Confidence
• S2DNAS:Transforming Static CNN Model for Dynamic Inference via Neural Architecture Search

Cited by in corpus (2)

• A Comprehensive Survey on Hardware-Aware Neural Architecture Search
• EBJR: Energy-Based Joint Reasoning for Adaptive Inference