Sparsely Aggregated Convolutional Networks

Abstract

We explore a key architectural aspect of deep convolutional neural networks: the pattern of internal skip connections used to aggregate outputs of earlier layers for consumption by deeper layers. Such aggregation is critical to facilitate training of very deep networks in an end-to-end manner. This is a primary reason for the widespread adoption of residual networks, which aggregate outputs via cumulative summation. While subsequent works investigate alternative aggregation operations (e.g. concatenation), we focus on an orthogonal question: which outputs to aggregate at a particular point in the network. We propose a new internal connection structure which aggregates only a sparse set of previous outputs at any given depth. Our experiments demonstrate this simple design change offers superior performance with fewer parameters and lower computational requirements. Moreover, we show that sparse aggregation allows networks to scale more robustly to 1000+ layers, thereby opening future avenues for training long-running visual processes.

Accepted to ECCV 2018

References in corpus (9)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Neural Architecture Search with Reinforcement Learning
• Compressing Neural Networks with the Hashing Trick
• Revisiting Unreasonable Effectiveness of Data in Deep Learning Era
• Wider or Deeper: Revisiting the ResNet Model for Visual Recognition
• The Reversible Residual Network: Backpropagation Without Storing Activations
• Highway and Residual Networks learn Unrolled Iterative Estimation
• Dual Path Networks
• Log-DenseNet: How to Sparsify a DenseNet

Cited by in corpus (4)

• Multi-level Residual Networks from Dynamical Systems View
• Compacting Deep Neural Networks for Internet of Things: Methods and Applications
• Shift-based Primitives for Efficient Convolutional Neural Networks
• ResFPN: Residual Skip Connections in Multi-Resolution Feature Pyramid Networks for Accurate Dense Pixel Matching