Bag of Tricks for Image Classification with Convolutional Neural Networks

Abstract

Much of the recent progress made in image classification research can be credited to training procedure refinements, such as changes in data augmentations and optimization methods. In the literature, however, most refinements are either briefly mentioned as implementation details or only visible in source code. In this paper, we will examine a collection of such refinements and empirically evaluate their impact on the final model accuracy through ablation study. We will show that, by combining these refinements together, we are able to improve various CNN models significantly. For example, we raise ResNet-50's top-1 validation accuracy from 75.3% to 79.29% on ImageNet. We will also demonstrate that improvement on image classification accuracy leads to better transfer learning performance in other application domains such as object detection and semantic segmentation.

arXiv comment10 pages, 9 tables, 4 figures

References in corpus (18)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Distilling the Knowledge in a Neural Network
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Rethinking Atrous Convolution for Semantic Image Segmentation
• mixup: Beyond Empirical Risk Minimization
• Deep Residual Learning for Image Recognition
• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
• Densely Connected Convolutional Networks
• SGDR: Stochastic Gradient Descent with Warm Restarts
• Mixed Precision Training
• Fully Convolutional Networks for Semantic Segmentation
• Don't Decay the Learning Rate, Increase the Batch Size
• Rethinking the Inception Architecture for Computer Vision
• Aggregated Residual Transformations for Deep Neural Networks
• Pyramid Scene Parsing Network
• Highly Scalable Deep Learning Training System with Mixed-Precision: Training ImageNet in Four Minutes
• ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
• Context Encoding for Semantic Segmentation

Cited by in corpus (29)

• fastai: A Layered API for Deep Learning
• GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond
• Revisiting ResNets: Improved Training and Scaling Strategies
• Bag of Freebies for Training Object Detection Neural Networks
• Second-order Non-local Attention Networks for Person Re-identification
• Gaussian Differential Privacy
• Stochastic Gradient Methods with Layer-wise Adaptive Moments for Training of Deep Networks
• Back to Simplicity: How to Train Accurate BNNs from Scratch?
• Inferring pointwise diffusion properties of single trajectories with deep learning
• Towards Better Accuracy-efficiency Trade-offs: Divide and Co-training
• Non-Structured DNN Weight Pruning -- Is It Beneficial in Any Platform?
• Fixing the train-test resolution discrepancy
• A Baseline for Few-Shot Image Classification
• VOC-ReID: Vehicle Re-identification based on Vehicle-Orientation-Camera
• Connecting optical morphology, environment, and HI mass fraction for low-redshift galaxies using deep learning
• Dynamic Mini-batch SGD for Elastic Distributed Training: Learning in the Limbo of Resources
• VIPriors 1: Visual Inductive Priors for Data-Efficient Deep Learning Challenges
• Explicit Shape Encoding for Real-Time Instance Segmentation
• FeatherNets: Convolutional Neural Networks as Light as Feather for Face Anti-spoofing
• Testing Deep Learning Models for Image Analysis Using Object-Relevant Metamorphic Relations
• Inductive Guided Filter: Real-time Deep Image Matting with Weakly Annotated Masks on Mobile Devices
• EnTri: Ensemble Learning with Tri-level Representations for Explainable Scene Recognition
• Multimodal End-to-End Group Emotion Recognition using Cross-Modal Attention
• Global Context Networks
• LogAvgExp Provides a Principled and Performant Global Pooling Operator
• A Benchmark on Tricks for Large-scale Image Retrieval
• Multi-Scale Dual-Branch Fully Convolutional Network for Hand Parsing
• Learning Equivariant Representations
• DeepSquare: Boosting the Learning Power of Deep Convolutional Neural Networks with Elementwise Square Operators