SBNet: Sparse Blocks Network for Fast Inference

Abstract

Conventional deep convolutional neural networks (CNNs) apply convolution operators uniformly in space across all feature maps for hundreds of layers - this incurs a high computational cost for real-time applications. For many problems such as object detection and semantic segmentation, we are able to obtain a low-cost computation mask, either from a priori problem knowledge, or from a low-resolution segmentation network. We show that such computation masks can be used to reduce computation in the high-resolution main network. Variants of sparse activation CNNs have previously been explored on small-scale tasks and showed no degradation in terms of object classification accuracy, but often measured gains in terms of theoretical FLOPs without realizing a practical speed-up when compared to highly optimized dense convolution implementations. In this work, we leverage the sparsity structure of computation masks and propose a novel tiling-based sparse convolution algorithm. We verified the effectiveness of our sparse CNN on LiDAR-based 3D object detection, and we report significant wall-clock speed-ups compared to dense convolution without noticeable loss of accuracy.

10 pages, CVPR 2018

References in corpus (11)

• Distilling the Knowledge in a Neural Network
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Focal Loss for Dense Object Detection
• Speeding up Convolutional Neural Networks with Low Rank Expansions
• Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights
• Learning Structured Sparsity in Deep Neural Networks
• Channel Pruning for Accelerating Very Deep Neural Networks
• RON: Reverse Connection with Objectness Prior Networks for Object Detection
• More is Less: A More Complicated Network with Less Inference Complexity
• Cnvlutin2: Ineffectual-Activation-and-Weight-Free Deep Neural Network Computing
• Speeding up Convolutional Neural Networks By Exploiting the Sparsity of Rectifier Units

Cited by in corpus (5)

• CalibNet: Geometrically Supervised Extrinsic Calibration using 3D Spatial Transformer Networks
• Bringing AI To Edge: From Deep Learning's Perspective
• Dynamic Neural Networks: A Survey
• Sparse and Dense Data with CNNs: Depth Completion and Semantic Segmentation
• Non-parametric Memory for Spatio-Temporal Segmentation of Construction Zones for Self-Driving