Multi-Label Image Recognition with Graph Convolutional Networks

Abstract

The task of multi-label image recognition is to predict a set of object labels that present in an image. As objects normally co-occur in an image, it is desirable to model the label dependencies to improve the recognition performance. To capture and explore such important dependencies, we propose a multi-label classification model based on Graph Convolutional Network (GCN). The model builds a directed graph over the object labels, where each node (label) is represented by word embeddings of a label, and GCN is learned to map this label graph into a set of inter-dependent object classifiers. These classifiers are applied to the image descriptors extracted by another sub-net, enabling the whole network to be end-to-end trainable. Furthermore, we propose a novel re-weighted scheme to create an effective label correlation matrix to guide information propagation among the nodes in GCN. Experiments on two multi-label image recognition datasets show that our approach obviously outperforms other existing state-of-the-art methods. In addition, visualization analyses reveal that the classifiers learned by our model maintain meaningful semantic topology.

To appear at CVPR 2019 (Source codes have been released on https://github.com/chenzhaomin123/ML_GCN)

References in corpus (6)

• Efficient Estimation of Word Representations in Vector Space
• CNN: Single-label to Multi-label
• Deep Convolutional Ranking for Multilabel Image Annotation
• RPC: A Large-Scale Retail Product Checkout Dataset
• Order-Free RNN with Visual Attention for Multi-Label Classification
• Chest X-rays Classification: A Multi-Label and Fine-Grained Problem

Cited by in corpus (18)

• Graph-Based Deep Learning for Medical Diagnosis and Analysis: Past, Present and Future
• Scalable Graph Neural Networks via Bidirectional Propagation
• Disentangled Variational Autoencoder based Multi-Label Classification with Covariance-Aware Multivariate Probit Model
• BernNet: Learning Arbitrary Graph Spectral Filters via Bernstein Approximation
• GM-MLIC: Graph Matching based Multi-Label Image Classification
• Learning Category Correlations for Multi-label Image Recognition with Graph Networks
• Consensus-Aware Visual-Semantic Embedding for Image-Text Matching
• Cross-Modality Attention with Semantic Graph Embedding for Multi-Label Classification
• Accurate, Efficient and Scalable Training of Graph Neural Networks
• Evaluating Multi-label Classifiers with Noisy Labels
• Modeling Sentiment Dependencies with Graph Convolutional Networks for Aspect-level Sentiment Classification
• Large-Scale Object Detection in the Wild from Imbalanced Multi-Labels
• GraftNet: An Engineering Implementation of CNN for Fine-grained Multi-label Task
• Look, Read and Feel: Benchmarking Ads Understanding with Multimodal Multitask Learning
• Multi-Label Gold Asymmetric Loss Correction with Single-Label Regulators
• Learning and Exploiting Interclass Visual Correlations for Medical Image Classification
• Dynamic Graph Correlation Learning for Disease Diagnosis with Incomplete Labels
• Multi-Label Learning from Single Positive Labels