Order-Free RNN with Visual Attention for Multi-Label Classification

Abstract

In this paper, we propose the joint learning attention and recurrent neural network (RNN) models for multi-label classification. While approaches based on the use of either model exist (e.g., for the task of image captioning), training such existing network architectures typically require pre-defined label sequences. For multi-label classification, it would be desirable to have a robust inference process, so that the prediction error would not propagate and thus affect the performance. Our proposed model uniquely integrates attention and Long Short Term Memory (LSTM) models, which not only addresses the above problem but also allows one to identify visual objects of interests with varying sizes without the prior knowledge of particular label ordering. More importantly, label co-occurrence information can be jointly exploited by our LSTM model. Finally, by advancing the technique of beam search, prediction of multiple labels can be efficiently achieved by our proposed network model.

Accepted at 32nd AAAI Conference on Artificial Intelligence (AAAI-18)

Cited by in corpus (13)

• The Emerging Trends of Multi-Label Learning
• Learning to Discover Multi-Class Attentional Regions for Multi-Label Image Recognition
• Sewer-ML: A Multi-Label Sewer Defect Classification Dataset and Benchmark
• Multi-Label Image Recognition with Graph Convolutional Networks
• General Multi-label Image Classification with Transformers
• Learning Category Correlations for Multi-label Image Recognition with Graph Networks
• Cross-Modality Attention with Semantic Graph Embedding for Multi-Label Classification
• Multi-Label Image Classification with Contrastive Learning
• Adapting RNN Sequence Prediction Model to Multi-label Set Prediction
• Elucidating image-to-set prediction: An analysis of models, losses and datasets
• Diversity in Fashion Recommendation using Semantic Parsing
• Multi-Scale Label Relation Learning for Multi-Label Classification Using 1-Dimensional Convolutional Neural Networks
• Transformer-based Dual Relation Graph for Multi-label Image Recognition