Learning Compact Reward for Image Captioning
arXiv:2003.10925
Abstract
Adversarial learning has shown its advances in generating natural and diverse descriptions in image captioning. However, the learned reward of existing adversarial methods is vague and ill-defined due to the reward ambiguity problem. In this paper, we propose a refined Adversarial Inverse Reinforcement Learning (rAIRL) method to handle the reward ambiguity problem by disentangling reward for each word in a sentence, as well as achieve stable adversarial training by refining the loss function to shift the generator towards Nash equilibrium. In addition, we introduce a conditional term in the loss function to mitigate mode collapse and to increase the diversity of the generated descriptions. Our experiments on MS COCO and Flickr30K show that our method can learn compact reward for image captioning.
arXiv comment13 pages, 10 figures
References in corpus (7)
- Conditional Generative Adversarial Nets
- Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
- Microsoft COCO Captions: Data Collection and Evaluation Server
- Improved Image Captioning via Policy Gradient optimization of SPIDEr
- A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Models
- Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow
- Contrastive Learning for Image Captioning