Gradually Updated Neural Networks for Large-Scale Image Recognition
arXiv:1711.09280
Abstract
Depth is one of the keys that make neural networks succeed in the task of large-scale image recognition. The state-of-the-art network architectures usually increase the depths by cascading convolutional layers or building blocks. In this paper, we present an alternative method to increase the depth. Our method is by introducing computation orderings to the channels within convolutional layers or blocks, based on which we gradually compute the outputs in a channel-wise manner. The added orderings not only increase the depths and the learning capacities of the networks without any additional computation costs, but also eliminate the overlap singularities so that the networks are able to converge faster and perform better. Experiments show that the networks based on our method achieve the state-of-the-art performances on CIFAR and ImageNet datasets.
References in corpus (15)
- mixup: Beyond Empirical Risk Minimization
- Deep Residual Learning for Image Recognition
- Striving for Simplicity: The All Convolutional Net
- Densely Connected Convolutional Networks
- Wide Residual Networks
- Training Very Deep Networks
- FractalNet: Ultra-Deep Neural Networks without Residuals
- Visualizing and Understanding Convolutional Networks
- Shake-Shake regularization
- Holistically-Nested Edge Detection
- Deep Networks with Stochastic Depth
- Skip Connections Eliminate Singularities
- Deconstructing the Ladder Network Architecture
- Dual Path Networks
- ScaleNet: Guiding Object Proposal Generation in Supermarkets and Beyond