Generating Diverse and Meaningful Captions
arXiv:1812.08126 · doi:10.1007/978-3-030-01418-6_18
Abstract
Image Captioning is a task that requires models to acquire a multi-modal understanding of the world and to express this understanding in natural language text. While the state-of-the-art for this task has rapidly improved in terms of n-gram metrics, these models tend to output the same generic captions for similar images. In this work, we address this limitation and train a model that generates more diverse and specific captions through an unsupervised training approach that incorporates a learning signal from an Image Retrieval model. We summarize previous results and improve the state-of-the-art on caption diversity and novelty. We make our source code publicly available online.
Accepted for presentation at The 27th International Conference on Artificial Neural Networks (ICANN 2018)
References in corpus (8)
- Adam: A Method for Stochastic Optimization
- Categorical Reparameterization with Gumbel-Softmax
- Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
- Microsoft COCO Captions: Data Collection and Evaluation Server
- Show and Tell: Lessons learned from the 2015 MSCOCO Image Captioning Challenge
- SentEval: An Evaluation Toolkit for Universal Sentence Representations
- Contrastive Learning for Image Captioning
- Diverse and Accurate Image Description Using a Variational Auto-Encoder with an Additive Gaussian Encoding Space