Cheaper Pre-training Lunch: An Efficient Paradigm for Object Detection
arXiv:2004.12178
Abstract
In this paper, we propose a general and efficient pre-training paradigm, Montage pre-training, for object detection. Montage pre-training needs only the target detection dataset while taking only 1/4 computational resources compared to the widely adopted ImageNet pre-training.To build such an efficient paradigm, we reduce the potential redundancy by carefully extracting useful samples from the original images, assembling samples in a Montage manner as input, and using an ERF-adaptive dense classification strategy for model pre-training. These designs include not only a new input pattern to improve the spatial utilization but also a novel learning objective to expand the effective receptive field of the pretrained model. The efficiency and effectiveness of Montage pre-training are validated by extensive experiments on the MS-COCO dataset, where the results indicate that the models using Montage pre-training are able to achieve on-par or even better detection performances compared with the ImageNet pre-training.
arXiv commentAccepted by ECCV2020
References in corpus (10)
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Momentum Contrast for Unsupervised Visual Representation Learning
- Understanding the Effective Receptive Field in Deep Convolutional Neural Networks
- EfficientDet: Scalable and Efficient Object Detection
- DetNet: A Backbone network for Object Detection
- Self-training with Noisy Student improves ImageNet classification
- Towards Accurate One-Stage Object Detection with AP-Loss
- Efficient Neural Architecture Transformation Searchin Channel-Level for Object Detection
- Computation Reallocation for Object Detection