Deep Pyramidal Residual Networks

Abstract

Deep convolutional neural networks (DCNNs) have shown remarkable performance in image classification tasks in recent years. Generally, deep neural network architectures are stacks consisting of a large number of convolutional layers, and they perform downsampling along the spatial dimension via pooling to reduce memory usage. Concurrently, the feature map dimension (i.e., the number of channels) is sharply increased at downsampling locations, which is essential to ensure effective performance because it increases the diversity of high-level attributes. This also applies to residual networks and is very closely related to their performance. In this research, instead of sharply increasing the feature map dimension at units that perform downsampling, we gradually increase the feature map dimension at all units to involve as many locations as possible. This design, which is discussed in depth together with our new insights, has proven to be an effective means of improving generalization ability. Furthermore, we propose a novel residual unit capable of further improving the classification accuracy with our new network architecture. Experiments on benchmark CIFAR-10, CIFAR-100, and ImageNet datasets have shown that our network architecture has superior generalization ability compared to the original residual networks. Code is available at https://github.com/jhkim89/PyramidNet}

Accepted to CVPR 2017

References in corpus (10)

• Striving for Simplicity: The All Convolutional Net
• FitNets: Hints for Thin Deep Nets
• Densely Connected Convolutional Networks
• Wide Residual Networks
• Going Deeper with Convolutions
• Fully Convolutional Networks for Semantic Segmentation
• FractalNet: Ultra-Deep Neural Networks without Residuals
• Residual Networks Behave Like Ensembles of Relatively Shallow Networks
• Fractional Max-Pooling
• Swapout: Learning an ensemble of deep architectures

Cited by in corpus (14)

• SGDR: Stochastic Gradient Descent with Warm Restarts
• Averaging Weights Leads to Wider Optima and Better Generalization
• Sharpness-Aware Minimization for Efficiently Improving Generalization
• Deep Convolutional Neural Network Design Patterns
• GResNet: Graph Residual Network for Reviving Deep GNNs from Suspended Animation
• Deep Pyramidal Residual Networks with Separated Stochastic Depth
• Content-Based Video-Music Retrieval Using Soft Intra-Modal Structure Constraint
• Learnable Image Encryption
• StackMix: A complementary Mix algorithm
• Analog-to-digital conversion revolutionized by deep learning
• Improving the Resolution of CNN Feature Maps Efficiently with Multisampling
• LogAvgExp Provides a Principled and Performant Global Pooling Operator
• Tandem Blocks in Deep Convolutional Neural Networks
• SwGridNet: A Deep Convolutional Neural Network based on Grid Topology for Image Classification