Hybrid Composition with IdleBlock: More Efficient Networks for Image Recognition
arXiv:1911.08609
Abstract
We propose a new building block, IdleBlock, which naturally prunes connections within the block. To fully utilize the IdleBlock we break the tradition of monotonic design in state-of-the-art networks, and introducing hybrid composition with IdleBlock. We study hybrid composition on MobileNet v3 and EfficientNet-B0, two of the most efficient networks. Without any neural architecture search, the deeper "MobileNet v3" with hybrid composition design surpasses possibly all state-of-the-art image recognition network designed by human experts or neural architecture search algorithms. Similarly, the hybridized EfficientNet-B0 networks are more efficient than previous state-of-the-art networks with similar computation budgets. These results suggest a new simpler and more efficient direction for network design and neural architecture search.
References in corpus (10)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
- mixup: Beyond Empirical Risk Minimization
- MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
- GluonCV and GluonNLP: Deep Learning in Computer Vision and Natural Language Processing
- Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolution
- Exploring Randomly Wired Neural Networks for Image Recognition
- Deep High-Resolution Representation Learning for Human Pose Estimation
- Dynamic Mini-batch SGD for Elastic Distributed Training: Learning in the Limbo of Resources