An Analysis of Pre-Training on Object Detection

Abstract

We provide a detailed analysis of convolutional neural networks which are pre-trained on the task of object detection. To this end, we train detectors on large datasets like OpenImagesV4, ImageNet Localization and COCO. We analyze how well their features generalize to tasks like image classification, semantic segmentation and object detection on small datasets like PASCAL-VOC, Caltech-256, SUN-397, Flowers-102 etc. Some important conclusions from our analysis are --- 1) Pre-training on large detection datasets is crucial for fine-tuning on small detection datasets, especially when precise localization is needed. For example, we obtain 81.1% mAP on the PASCAL-VOC dataset at 0.7 IoU after pre-training on OpenImagesV4, which is 7.6% better than the recently proposed DeformableConvNetsV2 which uses ImageNet pre-training. 2) Detection pre-training also benefits other localization tasks like semantic segmentation but adversely affects image classification. 3) Features for images (like avg. pooled Conv5) which are similar in the object detection feature space are likely to be similar in the image classification feature space but the converse is not true. 4) Visualization of features reveals that detection neurons have activations over an entire object, while activations for classification networks typically focus on parts. Therefore, detection networks are poor at classification when multiple instances are present in an image or when an instance only covers a small fraction of an image.

References in corpus (10)

• R-FCN: Object Detection via Region-based Fully Convolutional Networks
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Striving for Simplicity: The All Convolutional Net
• Understanding Neural Networks Through Deep Visualization
• ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness
• Visualizing Deep Convolutional Neural Networks Using Natural Pre-Images
• The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale
• What makes ImageNet good for transfer learning?
• Deformable ConvNets v2: More Deformable, Better Results
• Rethinking ImageNet Pre-training

Cited by in corpus (15)

• Rethinking Pre-training and Self-training
• A Primer on Motion Capture with Deep Learning: Principles, Pitfalls and Perspectives
• On Warm-Starting Neural Network Training
• Dense Contrastive Learning for Self-Supervised Visual Pre-Training
• Empirical Upper Bound in Object Detection and More
• Transfer Learning Gaussian Anomaly Detection by Fine-tuning Representations
• Zero-Shot Learning from scratch (ZFS): leveraging local compositional representations
• Rethinking Training from Scratch for Object Detection
• Deep Regionlets: Blended Representation and Deep Learning for Generic Object Detection
• Multi-Task Self-Training for Learning General Representations
• Neural Data Server: A Large-Scale Search Engine for Transfer Learning Data
• Understanding the Effects of Pre-Training for Object Detectors via Eigenspectrum
• GAIA: A Transfer Learning System of Object Detection that Fits Your Needs
• Empirical Upper Bound, Error Diagnosis and Invariance Analysis of Modern Object Detectors
• DAP: Detection-Aware Pre-training with Weak Supervision