Efficient DETR: Improving End-to-End Object Detector with Dense Prior
arXiv:2104.01318
Abstract
The recently proposed end-to-end transformer detectors, such as DETR and Deformable DETR, have a cascade structure of stacking 6 decoder layers to update object queries iteratively, without which their performance degrades seriously. In this paper, we investigate that the random initialization of object containers, which include object queries and reference points, is mainly responsible for the requirement of multiple iterations. Based on our findings, we propose Efficient DETR, a simple and efficient pipeline for end-to-end object detection. By taking advantage of both dense detection and sparse set detection, Efficient DETR leverages dense prior to initialize the object containers and brings the gap of the 1-decoder structure and 6-decoder structure. Experiments conducted on MS COCO show that our method, with only 3 encoder layers and 1 decoder layer, achieves competitive performance with state-of-the-art object detection methods. Efficient DETR is also robust in crowded scenes. It outperforms modern detectors on CrowdHuman dataset by a large margin.
10 pages, 5 figures, 10 tables
References in corpus (10)
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- YOLOv3: An Incremental Improvement
- R-FCN: Object Detection via Region-based Fully Convolutional Networks
- Deformable DETR: Deformable Transformers for End-to-End Object Detection
- CrowdHuman: A Benchmark for Detecting Human in a Crowd
- DenseBox: Unifying Landmark Localization with End to End Object Detection
- FreeAnchor: Learning to Match Anchors for Visual Object Detection
- AutoAssign: Differentiable Label Assignment for Dense Object Detection
- Sparse R-CNN: End-to-End Object Detection with Learnable Proposals
- Cascade RPN: Delving into High-Quality Region Proposal Network with Adaptive Convolution