Accelerating Deep Neural Networks with Spatial Bottleneck Modules
arXiv:1809.02601
Abstract
This paper presents an efficient module named spatial bottleneck for accelerating the convolutional layers in deep neural networks. The core idea is to decompose convolution into two stages, which first reduce the spatial resolution of the feature map, and then restore it to the desired size. This operation decreases the sampling density in the spatial domain, which is independent yet complementary to network acceleration approaches in the channel domain. Using different sampling rates, we can tradeoff between recognition accuracy and model complexity. As a basic building block, spatial bottleneck can be used to replace any single convolutional layer, or the combination of two convolutional layers. We empirically verify the effectiveness of spatial bottleneck by applying it to the deep residual networks. Spatial bottleneck achieves 2x and 1.4x speedup on the regular and channel-bottlenecked residual blocks, respectively, with the accuracies retained in recognizing low-resolution images, and even improved in recognizing high-resolution images.
arXiv comment9 pages, 5 figures
References in corpus (11)
- Neural Architecture Search with Reinforcement Learning
- DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
- DeepPose: Human Pose Estimation via Deep Neural Networks
- DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
- CNN Features off-the-shelf: an Astounding Baseline for Recognition
- Stacked Hourglass Networks for Human Pose Estimation
- Channel Pruning for Accelerating Very Deep Neural Networks
- Holistically-Nested Edge Detection
- Deeply-Fused Nets
- Deep Convolutional Neural Networks with Merge-and-Run Mappings
- Dual Path Networks