CondenseNet: An Efficient DenseNet using Learned Group Convolutions
arXiv:1711.09224
Abstract
Deep neural networks are increasingly used on mobile devices, where computational resources are limited. In this paper we develop CondenseNet, a novel network architecture with unprecedented efficiency. It combines dense connectivity with a novel module called learned group convolution. The dense connectivity facilitates feature re-use in the network, whereas learned group convolutions remove connections between layers for which this feature re-use is superfluous. At test time, our model can be implemented using standard group convolutions, allowing for efficient computation in practice. Our experiments show that CondenseNets are far more efficient than state-of-the-art compact convolutional networks such as MobileNets and ShuffleNets.
References in corpus (22)
- Distilling the Knowledge in a Neural Network
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
- Deep Residual Learning for Image Recognition
- Striving for Simplicity: The All Convolutional Net
- FitNets: Hints for Thin Deep Nets
- Wide Residual Networks
- SGDR: Stochastic Gradient Descent with Warm Restarts
- Going Deeper with Convolutions
- Training Very Deep Networks
- Binarized Neural Networks
- Pruning Filters for Efficient ConvNets
- FractalNet: Ultra-Deep Neural Networks without Residuals
- Compressing Neural Networks with the Hashing Trick
- Aggregated Residual Transformations for Deep Neural Networks
- Xception: Deep Learning with Depthwise Separable Convolutions
- Channel Pruning for Accelerating Very Deep Neural Networks
- Adaptive Computation Time for Recurrent Neural Networks
- Deep Networks with Stochastic Depth
- Interleaved Group Convolutions for Deep Neural Networks
- Deep Convolutional Neural Networks with Merge-and-Run Mappings
- Spatially Adaptive Computation Time for Residual Networks
Cited by in corpus (41)
- Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting
- Deep Learning Based Text Classification: A Comprehensive Review
- Knowledge Distillation by On-the-Fly Native Ensemble
- Big Bird: Transformers for Longer Sequences
- ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
- Path-Level Network Transformation for Efficient Architecture Search
- MONAS: Multi-Objective Neural Architecture Search using Reinforcement Learning
- Benchmarking Neural Network Robustness to Common Corruptions and Surface Variations
- Packing Sparse Convolutional Neural Networks for Efficient Systolic Array Implementations: Column Combining Under Joint Optimization
- A Programmable Approach to Neural Network Compression
- DPP-Net: Device-aware Progressive Search for Pareto-optimal Neural Architectures
- ChamNet: Towards Efficient Network Design through Platform-Aware Model Adaptation
- Blockwise Self-Attention for Long Document Understanding
- Selfish Sparse RNN Training
- Anytime Inference with Distilled Hierarchical Neural Ensembles
- The Power of Selecting Key Blocks with Local Pre-ranking for Long Document Information Retrieval
- Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Masks
- TT-Rec: Tensor Train Compression for Deep Learning Recommendation Models
- Efficient Forward Architecture Search
- Scatterbrain: Unifying Sparse and Low-rank Attention Approximation
- S2RMs: Spatially Structured Recurrent Modules
- SparseRT: Accelerating Unstructured Sparsity on GPUs for Deep Learning Inference
- HSD-CNN: Hierarchically self decomposing CNN architecture using class specific filter sensitivity analysis
- Term Revealing: Furthering Quantization at Run Time on Quantized DNNs
- Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Models
- Training Deep Neural Networks with Joint Quantization and Pruning of Weights and Activations
- AOGNets: Compositional Grammatical Architectures for Deep Learning
- Set-to-Sequence Methods in Machine Learning: a Review
- Accelerating SpMM Kernel with Cache-First Edge Sampling for Graph Neural Networks
- FlexSA: Flexible Systolic Array Architecture for Efficient Pruned DNN Model Training
- Low-Power Computer Vision: Status, Challenges, Opportunities
- Stochastic Gradient Descent with Hyperbolic-Tangent Decay on Classification
- SparseDNN: Fast Sparse Deep Learning Inference on CPUs
- ELASTIC: Improving CNNs with Dynamic Scaling Policies
- DECORE: Deep Compression with Reinforcement Learning
- Efficient Semantic Segmentation using Gradual Grouping
- Accelerated CNN Training Through Gradient Approximation
- Neural Networks at a Fraction with Pruned Quaternions
- ESPN: Extremely Sparse Pruned Networks
- Semantically Selective Augmentation for Deep Compact Person Re-Identification
- Full-stack Optimization for Accelerating CNNs with FPGA Validation