Channel Gating Neural Networks

Abstract

This paper introduces channel gating, a dynamic, fine-grained, and hardware-efficient pruning scheme to reduce the computation cost for convolutional neural networks (CNNs). Channel gating identifies regions in the features that contribute less to the classification result, and skips the computation on a subset of the input channels for these ineffective regions. Unlike static network pruning, channel gating optimizes CNN inference at run-time by exploiting input-specific characteristics, which allows substantially reducing the compute cost with almost no accuracy loss. We experimentally show that applying channel gating in state-of-the-art networks achieves 2.7-8.0$\times$ reduction in floating-point operations (FLOPs) and 2.0-4.4$\times$ reduction in off-chip memory accesses with a minimal accuracy loss on CIFAR-10. Combining our method with knowledge distillation reduces the compute cost of ResNet-18 by 2.6$\times$ without accuracy drop on ImageNet. We further demonstrate that channel gating can be realized in hardware efficiently. Our approach exhibits sparsity patterns that are well-suited to dense systolic arrays with minimal additional hardware. We have designed an accelerator for channel gating networks, which can be implemented using either FPGAs or ASICs. Running a quantized ResNet-18 model for ImageNet, our accelerator achieves an encouraging speedup of 2.4$\times$ on average, with a theoretical FLOP reduction of 2.8$\times$.

Cited by in corpus (29)

• Dynamic Channel Pruning: Feature Boosting and Suppression
• Bringing AI To Edge: From Deep Learning's Perspective
• Dynamic Feature Integration for Simultaneous Detection of Salient Object, Edge and Skeleton
• DAIS: Automatic Channel Pruning via Differentiable Annealing Indicator Search
• Pruning Algorithms to Accelerate Convolutional Neural Networks for Edge Applications: A Survey
• Dynamic Neural Networks: A Survey
• Glance and Focus: a Dynamic Approach to Reducing Spatial Redundancy in Image Classification
• Autoencoding with a Classifier System
• Locally Free Weight Sharing for Network Width Search
• AdaFuse: Adaptive Temporal Fusion Network for Efficient Action Recognition
• SegBlocks: Block-Based Dynamic Resolution Networks for Real-Time Segmentation
• S2DNAS:Transforming Static CNN Model for Dynamic Inference via Neural Architecture Search
• Faster Neural Network Training with Approximate Tensor Operations
• A Spike in Performance: Training Hybrid-Spiking Neural Networks with Quantized Activation Functions
• Computation on Sparse Neural Networks: an Inspiration for Future Hardware
• VA-RED${}^2$: Video Adaptive Redundancy Reduction
• Manifold Regularized Dynamic Network Pruning
• Learning to Compose Hypercolumns for Visual Correspondence
• Dynamic Group Convolution for Accelerating Convolutional Neural Networks
• Dynamic Network Quantization for Efficient Video Inference
• DS-Net++: Dynamic Weight Slicing for Efficient Inference in CNNs and Transformers
• BCNet: Searching for Network Width with Bilaterally Coupled Network
• DASNet: Dynamic Activation Sparsity for Neural Network Efficiency Improvement
• Adaptive Pixel-wise Structured Sparse Network for Efficient CNNs
• Learning Multiple Stock Trading Patterns with Temporal Routing Adaptor and Optimal Transport
• Dynamic Slimmable Network
• Dynamic Slimmable Denoising Network
• Gator: Customizable Channel Pruning of Neural Networks with Gating
• Enabling Design Methodologies and Future Trends for Edge AI: Specialization and Co-design