CNN-RNN: A Unified Framework for Multi-label Image Classification

Abstract

While deep convolutional neural networks (CNNs) have shown a great success in single-label image classification, it is important to note that real world images generally contain multiple labels, which could correspond to different objects, scenes, actions and attributes in an image. Traditional approaches to multi-label image classification learn independent classifiers for each category and employ ranking or thresholding on the classification results. These techniques, although working well, fail to explicitly exploit the label dependencies in an image. In this paper, we utilize recurrent neural networks (RNNs) to address this problem. Combined with CNNs, the proposed CNN-RNN framework learns a joint image-label embedding to characterize the semantic label dependency as well as the image-label relevance, and it can be trained end-to-end from scratch to integrate both information in a unified framework. Experimental results on public benchmark datasets demonstrate that the proposed architecture achieves better performance than the state-of-the-art multi-label classification model

CVPR 2016

References in corpus (10)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Sequence to Sequence Learning with Neural Networks
• Caffe: Convolutional Architecture for Fast Feature Embedding
• On the difficulty of training Recurrent Neural Networks
• Going Deeper with Convolutions
• CNN: Single-label to Multi-label
• CNN Features off-the-shelf: an Astounding Baseline for Recognition
• Deep Convolutional Ranking for Multilabel Image Annotation
• A Distributed Approach towards Discriminative Distance Metric Learning
• Compositional Distributional Semantics with Long Short Term Memory

Cited by in corpus (21)

• Multi-label Image Recognition by Recurrently Discovering Attentional Regions
• Learning Deep Latent Spaces for Multi-Label Classification
• Learning in an Uncertain World: Representing Ambiguity Through Multiple Hypotheses
• Learning Semantic-Specific Graph Representation for Multi-Label Image Recognition
• Deep Learning with a Rethinking Structure for Multi-label Classification
• GM-MLIC: Graph Matching based Multi-Label Image Classification
• What's Mine is Yours: Pretrained CNNs for Limited Training Sonar ATR
• Annotation Order Matters: Recurrent Image Annotator for Arbitrary Length Image Tagging
• Learning Category Correlations for Multi-label Image Recognition with Graph Networks
• Multi-Label Image Classification with Regional Latent Semantic Dependencies
• Cross-Modality Attention with Semantic Graph Embedding for Multi-Label Classification
• A New Distance Measure for Non-Identical Data with Application to Image Classification
• Recurrent Attentional Reinforcement Learning for Multi-label Image Recognition
• A Unified Model with Structured Output for Fashion Images Classification
• Webly Supervised Image Classification with Metadata: Automatic Noisy Label Correction via Visual-Semantic Graph
• Object-Level Context Modeling For Scene Classification with Context-CNN
• Improving CNN-base Stock Trading By Considering Data Heterogeneity and Burst
• Look, Read and Feel: Benchmarking Ads Understanding with Multimodal Multitask Learning
• Image Annotation using Multi-Layer Sparse Coding
• Labeled Data Generation with Inexact Supervision
• Multi-Label Learning from Single Positive Labels