Automatic Description Generation from Images: A Survey of Models, Datasets, and Evaluation Measures
arXiv:1601.03896
Abstract
Automatic description generation from natural images is a challenging problem that has recently received a large amount of interest from the computer vision and natural language processing communities. In this survey, we classify the existing approaches based on how they conceptualize this problem, viz., models that cast description as either generation problem or as a retrieval problem over a visual or multimodal representational space. We provide a detailed review of existing models, highlighting their advantages and disadvantages. Moreover, we give an overview of the benchmark image datasets and the evaluation measures that have been developed to assess the quality of machine-generated image descriptions. Finally we extrapolate future directions in the area of automatic image description generation.
arXiv commentJournal of Artificial Intelligence Research 55, 409-442, 2016
References in corpus (11)
- Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models
- Building Natural-Language Generation Systems
- VQA: Visual Question Answering
- Deep Fragment Embeddings for Bidirectional Image Sentence Mapping
- Return of the Devil in the Details: Delving Deep into Convolutional Nets
- Are You Talking to a Machine? Dataset and Methods for Multilingual Image Question Answering
- Describing Videos by Exploiting Temporal Structure
- Ask Your Neurons: A Neural-based Approach to Answering Questions about Images
- Language Models for Image Captioning: The Quirks and What Works
- Learning like a Child: Fast Novel Visual Concept Learning from Sentence Descriptions of Images
- A Dataset for Movie Description