Towards Efficient Convolutional Network Models with Filter Distribution Templates
arXiv:2104.08446
Abstract
Increasing number of filters in deeper layers when feature maps are decreased is a widely adopted pattern in convolutional network design. It can be found in classical CNN architectures and in automatic discovered models. Even CNS methods commonly explore a selection of multipliers derived from this pyramidal pattern. We defy this practice by introducing a small set of templates consisting of easy to implement, intuitive and aggressive variations of the original pyramidal distribution of filters in VGG and ResNet architectures. Experiments on CIFAR, CINIC10 and TinyImagenet datasets show that models produced by our templates, are more efficient in terms of fewer parameters and memory needs.
References in corpus (9)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- Improved Regularization of Convolutional Neural Networks with Cutout
- A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions
- AutoSlim: Towards One-Shot Architecture Search for Channel Numbers
- Network Pruning via Transformable Architecture Search
- Gate Decorator: Global Filter Pruning Method for Accelerating Deep Convolutional Neural Networks
- What is the State of Neural Network Pruning?
- Smallify: Learning Network Size while Training