DelugeNets: Deep Networks with Efficient and Flexible Cross-layer Information Inflows

Abstract

Deluge Networks (DelugeNets) are deep neural networks which efficiently facilitate massive cross-layer information inflows from preceding layers to succeeding layers. The connections between layers in DelugeNets are established through cross-layer depthwise convolutional layers with learnable filters, acting as a flexible yet efficient selection mechanism. DelugeNets can propagate information across many layers with greater flexibility and utilize network parameters more effectively compared to ResNets, whilst being more efficient than DenseNets. Remarkably, a DelugeNet model with just model complexity of 4.31 GigaFLOPs and 20.2M network parameters, achieve classification errors of 3.76% and 19.02% on CIFAR-10 and CIFAR-100 dataset respectively. Moreover, DelugeNet-122 performs competitively to ResNet-200 on ImageNet dataset, despite costing merely half of the computations needed by the latter.

arXiv commentCode: https://github.com/xternalz/DelugeNets

References in corpus (21)

• Deep Learning in Neural Networks: An Overview
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Deep Residual Learning for Image Recognition
• Striving for Simplicity: The All Convolutional Net
• Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
• Densely Connected Convolutional Networks
• Wide Residual Networks
• Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
• Training Very Deep Networks
• FractalNet: Ultra-Deep Neural Networks without Residuals
• Rethinking the Inception Architecture for Computer Vision
• Speeding up Convolutional Neural Networks with Low Rank Expansions
• Some Improvements on Deep Convolutional Neural Network Based Image Classification
• Xception: Deep Learning with Depthwise Separable Convolutions
• Fractional Max-Pooling
• Recent Advances in Convolutional Neural Networks
• Deep Networks with Stochastic Depth
• Flattened Convolutional Neural Networks for Feedforward Acceleration
• Swapout: Learning an ensemble of deep architectures
• Deep Residual Networks with Exponential Linear Unit
• Highway and Residual Networks learn Unrolled Iterative Estimation