Fine-Grained Neural Architecture Search
arXiv:1911.07478
Abstract
We present an elegant framework of fine-grained neural architecture search (FGNAS), which allows to employ multiple heterogeneous operations within a single layer and can even generate compositional feature maps using several different base operations. FGNAS runs efficiently in spite of significantly large search space compared to other methods because it trains networks end-to-end by a stochastic gradient descent method. Moreover, the proposed framework allows to optimize the network under predefined resource constraints in terms of number of parameters, FLOPs and latency. FGNAS has been applied to two crucial applications in resource demanding computer vision tasks---large-scale image classification and image super-resolution---and demonstrates the state-of-the-art performance through flexible operation search and channel pruning.
References in corpus (17)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- Neural Architecture Search with Reinforcement Learning
- Improved Regularization of Convolutional Neural Networks with Cutout
- MobileNetV2: Inverted Residuals and Linear Bottlenecks
- Densely Connected Convolutional Networks
- Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
- Channel Pruning for Accelerating Very Deep Neural Networks
- ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware
- SNAS: Stochastic Neural Architecture Search
- Residual Dense Network for Image Super-Resolution
- Progressive Neural Architecture Search
- Discrimination-aware Channel Pruning for Deep Neural Networks
- Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluation
- NetAdapt: Platform-Aware Neural Network Adaptation for Mobile Applications
- More is Less: A More Complicated Network with Less Inference Complexity
- DPP-Net: Device-aware Progressive Search for Pareto-optimal Neural Architectures