Inverting and Understanding Object Detectors
arXiv:2106.13933
Abstract
As a core problem in computer vision, the performance of object detection has improved drastically in the past few years. Despite their impressive performance, object detectors suffer from a lack of interpretability. Visualization techniques have been developed and widely applied to introspect the decisions made by other kinds of deep learning models; however, visualizing object detectors has been underexplored. In this paper, we propose using inversion as a primary tool to understand modern object detectors and develop an optimization-based approach to layout inversion, allowing us to generate synthetic images recognized by trained detectors as containing a desired configuration of objects. We reveal intriguing properties of detectors by applying our layout inversion technique to a variety of modern object detectors, and further investigate them via validation experiments: they rely on qualitatively different features for classification and regression; they learn canonical motifs of commonly co-occurring objects; they use diff erent visual cues to recognize objects of varying sizes. We hope our insights can help practitioners improve object detectors.
arXiv commentPreprints
References in corpus (9)
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- YOLOv4: Optimal Speed and Accuracy of Object Detection
- YOLOv3: An Incremental Improvement
- Understanding Neural Networks Through Deep Visualization
- Visualizing and Understanding Recurrent Networks
- SmoothGrad: removing noise by adding noise
- Visualizing Deep Convolutional Neural Networks Using Natural Pre-Images
- Learning What and Where to Draw
- Multifaceted Feature Visualization: Uncovering the Different Types of Features Learned By Each Neuron in Deep Neural Networks