Deep Convolutional Ranking for Multilabel Image Annotation

Abstract

Multilabel image annotation is one of the most important challenges in computer vision with many real-world applications. While existing work usually use conventional visual features for multilabel annotation, features based on Deep Neural Networks have shown potential to significantly boost performance. In this work, we propose to leverage the advantage of such features and analyze key components that lead to better performances. Specifically, we show that a significant performance gain could be obtained by combining convolutional architectures with approximate top-$k$ ranking objectives, as thye naturally fit the multilabel tagging problem. Our experiments on the NUS-WIDE dataset outperforms the conventional visual features by about 10%, obtaining the best reported performance in the literature.

References in corpus (2)

• Improving neural networks by preventing co-adaptation of feature detectors
• Stochastic Pooling for Regularization of Deep Convolutional Neural Networks

Cited by in corpus (67)

• CNN: Single-label to Multi-label
• Deep Label Distribution Learning with Label Ambiguity
• Image Captioning with Semantic Attention
• CNN-RNN: A Unified Framework for Multi-label Image Classification
• Unsupervised Learning of Visual Representations using Videos
• ProjE: Embedding Projection for Knowledge Graph Completion
• Deep Semantic Ranking Based Hashing for Multi-Label Image Retrieval
• Deep convolutional networks for quality assessment of protein folds
• Sewer-ML: A Multi-Label Sewer Defect Classification Dataset and Benchmark
• Multi-Label Image Recognition with Graph Convolutional Networks
• Learning Spatial Regularization with Image-level Supervisions for Multi-label Image Classification
• Multi-label Image Recognition by Recurrently Discovering Attentional Regions
• Learning Deep Latent Spaces for Multi-Label Classification
• HD-CNN: Hierarchical Deep Convolutional Neural Network for Large Scale Visual Recognition
• Video2GIF: Automatic Generation of Animated GIFs from Video
• Can multi-label classification networks know what they don't know?
• Learning in an Uncertain World: Representing Ambiguity Through Multiple Hypotheses
• Tensor Normalization and Full Distribution Training
• Learning Semantic-Specific Graph Representation for Multi-Label Image Recognition
• Learning Semantic Concepts and Order for Image and Sentence Matching
• GM-MLIC: Graph Matching based Multi-Label Image Classification
• Multi-Instance Visual-Semantic Embedding
• Transitive Invariance for Self-supervised Visual Representation Learning
• Annotation Order Matters: Recurrent Image Annotator for Arbitrary Length Image Tagging
• Learning Category Correlations for Multi-label Image Recognition with Graph Networks
• Multi-Label Image Classification with Regional Latent Semantic Dependencies
• Recurrent Attentional Reinforcement Learning for Multi-label Image Recognition
• Deep Semantic Dictionary Learning for Multi-label Image Classification
• MMFashion: An Open-Source Toolbox for Visual Fashion Analysis
• An Effective Automatic Image Annotation Model Via Attention Model and Data Equilibrium
• Learning Structured Inference Neural Networks with Label Relations
• Learning Structured Semantic Embeddings for Visual Recognition
• Tagging like Humans: Diverse and Distinct Image Annotation
• Multi-Modal Multi-Scale Deep Learning for Large-Scale Image Annotation
• Context Aware Image Annotation in Active Learning
• Automatic Malware Description via Attribute Tagging and Similarity Embedding
• Tag Recommendation by Word-Level Tag Sequence Modeling
• Tag Prediction at Flickr: a View from the Darkroom
• Multi-label Ranking: Mining Multi-label and Label Ranking Data
• Attentional Multilabel Learning over Graphs: A Message Passing Approach
• Attention-Aware Noisy Label Learning for Image Classification
• Large-Scale Object Detection in the Wild from Imbalanced Multi-Labels
• Image Classification and Retrieval from User-Supplied Tags
• Coarse to Fine: Multi-label Image Classification with Global/Local Attention
• Improving Pairwise Ranking for Multi-label Image Classification
• Learning to Point and Count
• Learning Image Conditioned Label Space for Multilabel Classification
• Instance-Aware Graph Convolutional Network for Multi-Label Classification
• GraftNet: An Engineering Implementation of CNN for Fine-grained Multi-label Task
• Towards Coherent and Cohesive Long-form Text Generation
• Multi-Label Zero-Shot Human Action Recognition via Joint Latent Ranking Embedding
• Measuring and Predicting Tag Importance for Image Retrieval
• CPAR: Cloud-Assisted Privacy-preserving Image Annotation with Randomized KD-Forest
• Multi-label Classification with Optimal Thresholding for Multi-composition Spectroscopic Analysis
• Semi-Supervised Graph Embedding for Multi-Label Graph Node Classification
• Adversarial Learning of Label Dependency: A Novel Framework for Multi-class Classification
• Improving Classification by Improving Labelling: Introducing Probabilistic Multi-Label Object Interaction Recognition
• Tensor Composition Net for Visual Relationship Prediction
• Simple to Complex Cross-modal Learning to Rank
• Improving Multi-label Learning with Missing Labels by Structured Semantic Correlations
• Zero-Shot Learning with Sparse Attribute Propagation
• Unifying Relational Sentence Generation and Retrieval for Medical Image Report Composition
• Holistic and Comprehensive Annotation of Clinically Significant Findings on Diverse CT Images: Learning from Radiology Reports and Label Ontology
• Deep Unified Multimodal Embeddings for Understanding both Content and Users in Social Media Networks
• Prototype Matching Networks for Large-Scale Multi-label Genomic Sequence Classification
• Kill Two Birds with One Stone: Weakly-Supervised Neural Network for Image Annotation and Tag Refinement
• Image Annotation using Multi-Layer Sparse Coding