MemNet: Memory-Efficiency Guided Neural Architecture Search with Augment-Trim learning
arXiv:1907.09569
Abstract
Recent studies on automatic neural architectures search have demonstrated significant performance, competitive to or even better than hand-crafted neural architectures. However, most of the existing network architecture tend to use residual, parallel structures and concatenation block between shallow and deep features to construct a large network. This requires large amounts of memory for storing both weights and feature maps. This is challenging for mobile and embedded devices since they may not have enough memory to perform inference with the designed large network model. To close this gap, we propose MemNet, an augment-trim learning-based neural network search framework that optimizes not only performance but also memory requirement. Specifically, it employs memory consumption based ranking score which forces an upper bound on memory consumption for navigating the search process. Experiment results show that, as compared to the state-of-the-art efficient designing methods, MemNet can find an architecture which can achieve competitive accuracy and save an average of 24.17% on the total memory needed.
References in corpus (19)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
- Neural Architecture Search with Reinforcement Learning
- Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
- SGDR: Stochastic Gradient Descent with Warm Restarts
- Going Deeper with Convolutions
- DARTS: Differentiable Architecture Search
- Efficient Neural Architecture Search via Parameter Sharing
- Deep, Convolutional, and Recurrent Models for Human Activity Recognition using Wearables
- Efficient Architecture Search by Network Transformation
- ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware
- Progressive Neural Architecture Search
- MONAS: Multi-Objective Neural Architecture Search using Reinforcement Learning
- AutoSlim: Towards One-Shot Architecture Search for Channel Numbers
- You Only Search Once: Single Shot Neural Architecture Search via Direct Sparse Optimization
- KTAN: Knowledge Transfer Adversarial Network
- DVOLVER: Efficient Pareto-Optimal Neural Network Architecture Search
- Wearable Audio and IMU Based Shot Detection in Racquet Sports