What is Holding Back Convnets for Detection?

Abstract

Convolutional neural networks have recently shown excellent results in general object detection and many other tasks. Albeit very effective, they involve many user-defined design choices. In this paper we want to better understand these choices by inspecting two key aspects "what did the network learn?", and "what can the network learn?". We exploit new annotations (Pascal3D+), to enable a new empirical analysis of the R-CNN detector. Despite common belief, our results indicate that existing state-of-the-art convnet architectures are not invariant to various appearance factors. In fact, all considered networks have similar weak points which cannot be mitigated by simply increasing the training data (architectural changes are needed). We show that overall performance can improve when using image renderings for data augmentation. We report the best known results on the Pascal3D+ detection and view-point estimation tasks.

References in corpus (4)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Going Deeper with Convolutions
• Holistically-Nested Edge Detection
• Descriptor Matching with Convolutional Neural Networks: a Comparison to SIFT

Cited by in corpus (5)

• DenseBox: Unifying Landmark Localization with End to End Object Detection
• DeepFool: a simple and accurate method to fool deep neural networks
• How useful is photo-realistic rendering for visual learning?
• Convolutional Models for Joint Object Categorization and Pose Estimation
• Generative Model with Coordinate Metric Learning for Object Recognition Based on 3D Models