Exploit Bounding Box Annotations for Multi-label Object Recognition
arXiv:1504.05843
Abstract
Convolutional neural networks (CNNs) have shown great performance as general feature representations for object recognition applications. However, for multi-label images that contain multiple objects from different categories, scales and locations, global CNN features are not optimal. In this paper, we incorporate local information to enhance the feature discriminative power. In particular, we first extract object proposals from each image. With each image treated as a bag and object proposals extracted from it treated as instances, we transform the multi-label recognition problem into a multi-class multi-instance learning problem. Then, in addition to extracting the typical CNN feature representation from each proposal, we propose to make use of ground-truth bounding box annotations (strong labels) to add another level of local information by using nearest-neighbor relationships of local regions to form a multi-view pipeline. The proposed multi-view multi-instance framework utilizes both weak and strong labels effectively, and more importantly it has the generalization ability to even boost the performance of unseen categories by partial strong labels from other categories. Our framework is extensively compared with state-of-the-art hand-crafted feature based methods and CNN based methods on two multi-label benchmark datasets. The experimental results validate the discriminative power and the generalization ability of the proposed framework. With strong labels, our framework is able to achieve state-of-the-art results in both datasets.
arXiv commentAccepted in CVPR 2016
References in corpus (9)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks
- CNN: Single-label to Multi-label
- Return of the Devil in the Details: Delving Deep into Convolutional Nets
- CNN Features off-the-shelf: an Astounding Baseline for Recognition
- A Survey on Metric Learning for Feature Vectors and Structured Data
- Rich feature hierarchies for accurate object detection and semantic segmentation
- Top Rank Optimization in Linear Time
- How good are detection proposals, really?
Cited by in corpus (4)
- Learning Spatial Regularization with Image-level Supervisions for Multi-label Image Classification
- Learning in an Uncertain World: Representing Ambiguity Through Multiple Hypotheses
- Multi-label Pixelwise Classification for Reconstruction of Large-scale Urban Areas
- Improving Classification by Improving Labelling: Introducing Probabilistic Multi-Label Object Interaction Recognition