Understanding the Effects of Pre-Training for Object Detectors via Eigenspectrum

Abstract

ImageNet pre-training has been regarded as essential for training accurate object detectors for a long time. Recently, it has been shown that object detectors trained from randomly initialized weights can be on par with those fine-tuned from ImageNet pre-trained models. However, the effects of pre-training and the differences caused by pre-training are still not fully understood. In this paper, we analyze the eigenspectrum dynamics of the covariance matrix of each feature map in object detectors. Based on our analysis on ResNet-50, Faster R-CNN with FPN, and Mask R-CNN, we show that object detectors trained from ImageNet pre-trained models and those trained from scratch behave differently from each other even if both object detectors have similar accuracy. Furthermore, we propose a method for automatically determining the widths (the numbers of channels) of object detectors based on the eigenspectrum. We train Faster R-CNN with FPN from randomly initialized weights, and show that our method can reduce ~27% of the parameters of ResNet-50 without increasing Multiply-Accumulate operations and losing accuracy. Our results indicate that we should develop more appropriate methods for transferring knowledge from image classification to object detection (or other tasks).

arXiv commentICCV 2019 Workshop on Neural Architects (Oral)

References in corpus (22)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Overcoming catastrophic forgetting in neural networks
• Neural Architecture Search with Reinforcement Learning
• How transferable are features in deep neural networks?
• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
• DARTS: Differentiable Architecture Search
• AMC: AutoML for Model Compression and Acceleration on Mobile Devices
• MMDetection: Open MMLab Detection Toolbox and Benchmark
• Opening the Black Box of Deep Neural Networks via Information
• Transfusion: Understanding Transfer Learning for Medical Imaging
• Searching for Efficient Multi-Scale Architectures for Dense Image Prediction
• What makes ImageNet good for transfer learning?
• Deep Learning for Generic Object Detection: A Survey
• SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability
• DetNAS: Backbone Search for Object Detection
• Convolutional Neural Fabrics
• Differentiable Learning-to-Normalize via Switchable Normalization
• Tiny-DSOD: Lightweight Object Detection for Resource-Restricted Usages
• An Analysis of Pre-Training on Object Detection
• Spectral Pruning: Compressing Deep Neural Networks via Spectral Analysis and its Generalization Error