Training Deeper Convolutional Networks with Deep Supervision

Abstract

One of the most promising ways of improving the performance of deep convolutional neural networks is by increasing the number of convolutional layers. However, adding layers makes training more difficult and computationally expensive. In order to train deeper networks, we propose to add auxiliary supervision branches after certain intermediate layers during training. We formulate a simple rule of thumb to determine where these branches should be added. The resulting deeply supervised structure makes the training much easier and also produces better classification results on ImageNet and the recently released, larger MIT Places dataset

References in corpus (4)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Going Deeper with Convolutions
• Deeply-Supervised Nets

Cited by in corpus (28)

• Knowledge Guided Disambiguation for Large-Scale Scene Classification with Multi-Resolution CNNs
• Places205-VGGNet Models for Scene Recognition
• Augmenting Supervised Neural Networks with Unsupervised Objectives for Large-scale Image Classification
• Locally-Supervised Deep Hybrid Model for Scene Recognition
• Power Normalizations in Fine-grained Image, Few-shot Image and Graph Classification
• Attention-guided Chained Context Aggregation for Semantic Segmentation
• Deeply Supervised Skin Lesions Diagnosis with Stage and Branch Attention
• Accurate Retinal Vessel Segmentation via Octave Convolution Neural Network
• PINQI: An End-to-End Physics-Informed Approach to Learned Quantitative MRI Reconstruction
• A Densely Interconnected Network for Deep Learning Accelerated MRI
• On uncertainty estimation in active learning for image segmentation
• Multilevel Wavelet Decomposition Network for Interpretable Time Series Analysis
• Residual Squeeze VGG16
• The Shallow End: Empowering Shallower Deep-Convolutional Networks through Auxiliary Outputs
• Auxiliary Learning for Deep Multi-task Learning
• Neural networks for semantic segmentation of historical city maps: Cross-cultural performance and the impact of figurative diversity
• Collaborative Layer-wise Discriminative Learning in Deep Neural Networks
• Multi-output Deep-Supervised Classifier Chains for Plant Pathology
• Alleviating Class-wise Gradient Imbalance for Pulmonary Airway Segmentation
• Object-Level Context Modeling For Scene Classification with Context-CNN
• Depth Completion via Deep Basis Fitting
• Fine-Grained Energy and Performance Profiling framework for Deep Convolutional Neural Networks
• Energy Predictive Models for Convolutional Neural Networks on Mobile Platforms
• The Compressed Model of Residual CNDS
• A Deeper Look at Power Normalizations
• Beyond Single Stage Encoder-Decoder Networks: Deep Decoders for Semantic Image Segmentation
• Residual-CNDS for Grand Challenge Scene Dataset
• Efficient 2.5D Hand Pose Estimation via Auxiliary Multi-Task Training for Embedded Devices