Simple Image Description Generator via a Linear Phrase-Based Approach
arXiv:1412.8419
Abstract
Generating a novel textual description of an image is an interesting problem that connects computer vision and natural language processing. In this paper, we present a simple model that is able to generate descriptive sentences given a sample image. This model has a strong focus on the syntax of the descriptions. We train a purely bilinear model that learns a metric between an image representation (generated from a previously trained Convolutional Neural Network) and phrases that are used to described them. The system is then able to infer phrases from a given image sample. Based on caption syntax statistics, we propose a simple language model that can produce relevant descriptions for a given test image using the phrases inferred. Our approach, which is considerably simpler than state-of-the-art models, achieves comparable results on the recently release Microsoft COCO dataset.
Accepted as a workshop paper at ICLR 2015
References in corpus (7)
- Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models
- Return of the Devil in the Details: Delving Deep into Convolutional Nets
- Explain Images with Multimodal Recurrent Neural Networks
- Show and Tell: A Neural Image Caption Generator
- Deep Visual-Semantic Alignments for Generating Image Descriptions
- From Captions to Visual Concepts and Back
- Rehabilitation of Count-based Models for Word Vector Representations
Cited by in corpus (11)
- Microsoft COCO Captions: Data Collection and Evaluation Server
- Are You Talking to a Machine? Dataset and Methods for Multilingual Image Question Answering
- Image Captioning with Semantic Attention
- Exploring Nearest Neighbor Approaches for Image Captioning
- Towards Diverse and Natural Image Descriptions via a Conditional GAN
- Learning like a Child: Fast Novel Visual Concept Learning from Sentence Descriptions of Images
- Deep Reinforcement Learning-based Image Captioning with Embedding Reward
- A Comprehensive Survey of Deep Learning for Image Captioning
- "Factual" or "Emotional": Stylized Image Captioning with Adaptive Learning and Attention
- Figure Captioning with Reasoning and Sequence-Level Training
- Robust Image Captioning