LeanResNet: A Low-cost Yet Effective Convolutional Residual Networks

Abstract

Convolutional Neural Networks (CNNs) filter the input data using spatial convolution operators with compact stencils. Commonly, the convolution operators couple features from all channels, which leads to immense computational cost in the training of and prediction with CNNs. To improve the efficiency of CNNs, we introduce lean convolution operators that reduce the number of parameters and computational complexity, and can be used in a wide range of existing CNNs. Here, we exemplify their use in residual networks (ResNets), which have been very reliable for a few years now and analyzed intensively. In our experiments on three image classification problems, the proposed LeanResNet yields results that are comparable to other recently proposed reduced architectures using similar number of parameters.

References in corpus (7)

• Adam: A Method for Stochastic Optimization
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Pruning Convolutional Neural Networks for Resource Efficient Inference
• Pruning Filters for Efficient ConvNets
• DSD: Dense-Sparse-Dense Training for Deep Neural Networks
• The Power of Sparsity in Convolutional Neural Networks
• Design of Efficient Convolutional Layers using Single Intra-channel Convolution, Topological Subdivisioning and Spatial "Bottleneck" Structure

Cited by in corpus (1)

• Symmetric block-low-rank layers for fully reversible multilevel neural networks