Multi-label Image Recognition by Recurrently Discovering Attentional Regions

Abstract

This paper proposes a novel deep architecture to address multi-label image recognition, a fundamental and practical task towards general visual understanding. Current solutions for this task usually rely on an extra step of extracting hypothesis regions (i.e., region proposals), resulting in redundant computation and sub-optimal performance. In this work, we achieve the interpretable and contextualized multi-label image classification by developing a recurrent memorized-attention module. This module consists of two alternately performed components: i) a spatial transformer layer to locate attentional regions from the convolutional feature maps in a region-proposal-free way and ii) an LSTM (Long-Short Term Memory) sub-network to sequentially predict semantic labeling scores on the located regions while capturing the global dependencies of these regions. The LSTM also output the parameters for computing the spatial transformer. On large-scale benchmarks of multi-label image classification (e.g., MS-COCO and PASCAL VOC 07), our approach demonstrates superior performances over other existing state-of-the-arts in both accuracy and efficiency.

arXiv commentAccepted at ICCV 2017

References in corpus (10)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Multiple Object Recognition with Visual Attention
• CNN Features off-the-shelf: an Astounding Baseline for Recognition
• Deep Convolutional Ranking for Multilabel Image Annotation
• CNN-RNN: A Unified Framework for Multi-label Image Classification
• Recurrent Attentional Networks for Saliency Detection
• Multi-Label Image Classification with Regional Latent Semantic Dependencies

Cited by in corpus (16)

• Discriminative Feature Learning with Foreground Attention for Person Re-Identification
• Crowd Counting using Deep Recurrent Spatial-Aware Network
• Pedestrian Attribute Recognition: A Survey
• GM-MLIC: Graph Matching based Multi-Label Image Classification
• Learning Category Correlations for Multi-label Image Recognition with Graph Networks
• Cross-Modality Attention with Semantic Graph Embedding for Multi-Label Classification
• Knowledge-Embedded Representation Learning for Fine-Grained Image Recognition
• Tagging like Humans: Diverse and Distinct Image Annotation
• Large-Scale Object Detection in the Wild from Imbalanced Multi-Labels
• Contrast-Oriented Deep Neural Networks for Salient Object Detection
• GraftNet: An Engineering Implementation of CNN for Fine-grained Multi-label Task
• Unsupervised Multi-label Dataset Generation from Web Data
• Weakly Supervised Domain-Specific Color Naming Based on Attention
• Adversarial Learning of Label Dependency: A Novel Framework for Multi-class Classification
• Reconstruction Regularized Deep Metric Learning for Multi-label Image Classification
• Learning to Segment Object Candidates via Recursive Neural Networks