Rethinking ImageNet Pre-training

Abstract

We report competitive results on object detection and instance segmentation on the COCO dataset using standard models trained from random initialization. The results are no worse than their ImageNet pre-training counterparts even when using the hyper-parameters of the baseline system (Mask R-CNN) that were optimized for fine-tuning pre-trained models, with the sole exception of increasing the number of training iterations so the randomly initialized models may converge. Training from random initialization is surprisingly robust; our results hold even when: (i) using only 10% of the training data, (ii) for deeper and wider models, and (iii) for multiple tasks and metrics. Experiments show that ImageNet pre-training speeds up convergence early in training, but does not necessarily provide regularization or improve final target task accuracy. To push the envelope we demonstrate 50.9 AP on COCO object detection without using any external data---a result on par with the top COCO 2017 competition results that used ImageNet pre-training. These observations challenge the conventional wisdom of ImageNet pre-training for dependent tasks and we expect these discoveries will encourage people to rethink the current de facto paradigm of `pre-training and fine-tuning' in computer vision.

arXiv commentTechnical report

References in corpus (22)

• Two-Stream Convolutional Networks for Action Recognition in Videos
• Deep Residual Learning for Image Recognition
• DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
• Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
• Instance Normalization: The Missing Ingredient for Fast Stylization
• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
• Fully Convolutional Networks for Semantic Segmentation
• Rich feature hierarchies for accurate object detection and semantic segmentation
• Layer Normalization
• Feature Pyramid Networks for Object Detection
• Cascade R-CNN: Delving into High Quality Object Detection
• Aggregated Residual Transformations for Deep Neural Networks
• Path Aggregation Network for Instance Segmentation
• Revisiting Unreasonable Effectiveness of Data in Deep Learning Era
• Unsupervised Visual Representation Learning by Context Prediction
• Batch Renormalization: Towards Reducing Minibatch Dependence in Batch-Normalized Models
• DetNet: A Backbone network for Object Detection
• Unsupervised Learning of Visual Representations using Videos
• DSOD: Learning Deeply Supervised Object Detectors from Scratch
• Analyzing the Performance of Multilayer Neural Networks for Object Recognition
• MegDet: A Large Mini-Batch Object Detector
• Learning Features by Watching Objects Move

Cited by in corpus (53)

• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
• ResUNet-a: a deep learning framework for semantic segmentation of remotely sensed data
• MMDetection: Open MMLab Detection Toolbox and Benchmark
• VL-BERT: Pre-training of Generic Visual-Linguistic Representations
• Learning Representations by Maximizing Mutual Information Across Views
• High-Resolution Representations for Labeling Pixels and Regions
• Transfusion: Understanding Transfer Learning for Medical Imaging
• Object Detection in 20 Years: A Survey
• Prototypical Contrastive Learning of Unsupervised Representations
• Med3D: Transfer Learning for 3D Medical Image Analysis
• Billion-scale semi-supervised learning for image classification
• A Primer on Motion Capture with Deep Learning: Principles, Pitfalls and Perspectives
• What is being transferred in transfer learning?
• Cascade R-CNN: High Quality Object Detection and Instance Segmentation
• Test-Time Training with Self-Supervision for Generalization under Distribution Shifts
• Self-EMD: Self-Supervised Object Detection without ImageNet
• Gated CRF Loss for Weakly Supervised Semantic Image Segmentation
• Object DGCNN: 3D Object Detection using Dynamic Graphs
• Efficient Neural Architecture Transformation Searchin Channel-Level for Object Detection
• An Analysis of Pre-Training on Object Detection
• Improved Selective Refinement Network for Face Detection
• Characterizing soot in TEM images using a convolutional neural network
• FibeR-CNN: Expanding Mask R-CNN to Improve Image-Based Fiber Analysis
• Side-Aware Boundary Localization for More Precise Object Detection
• Leveraging Adaptive Color Augmentation in Convolutional Neural Networks for Deep Skin Lesion Segmentation
• OR-UNet: an Optimized Robust Residual U-Net for Instrument Segmentation in Endoscopic Images
• U-NetPlus: A Modified Encoder-Decoder U-Net Architecture for Semantic and Instance Segmentation of Surgical Instrument
• 3D convolutional neural network for abdominal aortic aneurysm segmentation
• BoLTVOS: Box-Level Tracking for Video Object Segmentation
• SM-NAS: Structural-to-Modular Neural Architecture Search for Object Detection
• Towards Understanding the Transferability of Deep Representations
• On the impact of selected modern deep-learning techniques to the performance and celerity of classification models in an experimental high-energy physics use case
• Benchmarking the Robustness of Instance Segmentation Models
• Towards Real-Time Action Recognition on Mobile Devices Using Deep Models
• Seesaw-Net: Convolution Neural Network With Uneven Group Convolution
• Neural Data Server: A Large-Scale Search Engine for Transfer Learning Data
• Non-binary deep transfer learning for image classification
• Improving Prognostic Performance in Resectable Pancreatic Ductal Adenocarcinoma using Radiomics and Deep Learning Features Fusion in CT Images
• FNA++: Fast Network Adaptation via Parameter Remapping and Architecture Search
• Large-Scale Object Detection in the Wild from Imbalanced Multi-Labels
• Learning an Efficient Network for Large-Scale Hierarchical Object Detection with Data Imbalance: 3rd Place Solution to Open Images Challenge 2019
• Towards Adversarially Robust Object Detection
• CATCH: Context-based Meta Reinforcement Learning for Transferrable Architecture Search
• Object Detection from Scratch with Deep Supervision
• Analysis of Big Data Technology for Health Care Services
• EHSOD: CAM-Guided End-to-end Hybrid-Supervised Object Detection with Cascade Refinement
• CRL: Class Representative Learning for Image Classification
• Real time backbone for semantic segmentation
• Global Context Networks
• Meta-learning Transferable Representations with a Single Target Domain
• Improve Unsupervised Pretraining for Few-label Transfer
• POD: Practical Object Detection with Scale-Sensitive Network
• Training few-shot classification via the perspective of minibatch and pretraining