Analysis of diversity-accuracy tradeoff in image captioning
arXiv:2002.11848
Abstract
We investigate the effect of different model architectures, training objectives, hyperparameter settings and decoding procedures on the diversity of automatically generated image captions. Our results show that 1) simple decoding by naive sampling, coupled with low temperature is a competitive and fast method to produce diverse and accurate caption sets; 2) training with CIDEr-based reward using Reinforcement learning harms the diversity properties of the resulting generator, which cannot be mitigated by manipulating decoding parameters. In addition, we propose a new metric AllSPICE for evaluating both accuracy and diversity of a set of captions by a single value.
References in corpus (9)
- Adam: A Method for Stochastic Optimization
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- The Curious Case of Neural Text Degeneration
- Hierarchical Neural Story Generation
- Variational inference for Monte Carlo objectives
- Mutual Information and Diverse Decoding Improve Neural Machine Translation
- Describing like humans: on diversity in image captioning
- Fast, Diverse and Accurate Image Captioning Guided By Part-of-Speech
- Unsupervised Recurrent Neural Network Grammars