An Efficient Quantitative Approach for Optimizing Convolutional Neural Networks

Abstract

With the increasing popularity of deep learning, Convolutional Neural Networks (CNNs) have been widely applied in various domains, such as image classification and object detection, and achieve stunning success in terms of their high accuracy over the traditional statistical methods. To exploit the potential of CNN models, a huge amount of research and industry efforts have been devoted to optimizing CNNs. Among these endeavors, CNN architecture design has attracted tremendous attention because of its great potential of improving model accuracy or reducing model complexity. However, existing work either introduces repeated training overhead in the search process or lacks an interpretable metric to guide the design. To clear these hurdles, we propose 3D-Receptive Field (3DRF), an explainable and easy-to-compute metric, to estimate the quality of a CNN architecture and guide the search process of designs. To validate the effectiveness of 3DRF, we build a static optimizer to improve the CNN architectures at both the stage level and the kernel level. Our optimizer not only provides a clear and reproducible procedure but also mitigates unnecessary training efforts in the architecture search process. Extensive experiments and studies show that the models generated by our optimizer can achieve up to 5.47% accuracy improvement and up to 65.38% parameters deduction, compared with state-of-the-art CNN structures like MobileNet and ResNet.

References in corpus (12)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Fully Convolutional Networks for Semantic Segmentation
• Pruning Filters for Efficient ConvNets
• Efficient Neural Architecture Search via Parameter Sharing
• Large-Scale Evolution of Image Classifiers
• Designing Neural Network Architectures using Reinforcement Learning
• Learning Efficient Convolutional Networks through Network Slimming
• Simple And Efficient Architecture Search for Convolutional Neural Networks
• AtomNAS: Fine-Grained End-to-End Neural Architecture Search
• BigNAS: Scaling Up Neural Architecture Search with Big Single-Stage Models
• RC-DARTS: Resource Constrained Differentiable Architecture Search