Annotation Order Matters: Recurrent Image Annotator for Arbitrary Length Image Tagging
arXiv:1604.05225
Abstract
Automatic image annotation has been an important research topic in facilitating large scale image management and retrieval. Existing methods focus on learning image-tag correlation or correlation between tags to improve annotation accuracy. However, most of these methods evaluate their performance using top-k retrieval performance, where k is fixed. Although such setting gives convenience for comparing different methods, it is not the natural way that humans annotate images. The number of annotated tags should depend on image contents. Inspired by the recent progress in machine translation and image captioning, we propose a novel Recurrent Image Annotator (RIA) model that forms image annotation task as a sequence generation problem so that RIA can natively predict the proper length of tags according to image contents. We evaluate the proposed model on various image annotation datasets. In addition to comparing our model with existing methods using the conventional top-k evaluation measures, we also provide our model as a high quality baseline for the arbitrary length image tagging task. Moreover, the results of our experiments show that the order of tags in training phase has a great impact on the final annotation performance.
arXiv commentInternational Conference on Pattern Recognition (ICPR) 2016, Cancun, Mexico (oral presentation)
References in corpus (9)
- Adam: A Method for Stochastic Optimization
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Efficient Estimation of Word Representations in Vector Space
- Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
- On the difficulty of training Recurrent Neural Networks
- CNN: Single-label to Multi-label
- Explain Images with Multimodal Recurrent Neural Networks
- Deep Convolutional Ranking for Multilabel Image Annotation
- CNN-RNN: A Unified Framework for Multi-label Image Classification