Actor-Critic Sequence Training for Image Captioning

Abstract

Generating natural language descriptions of images is an important capability for a robot or other visual-intelligence driven AI agent that may need to communicate with human users about what it is seeing. Such image captioning methods are typically trained by maximising the likelihood of ground-truth annotated caption given the image. While simple and easy to implement, this approach does not directly maximise the language quality metrics we care about such as CIDEr. In this paper we investigate training image captioning methods based on actor-critic reinforcement learning in order to directly optimise non-differentiable quality metrics of interest. By formulating a per-token advantage and value computation strategy in this novel reinforcement learning based captioning model, we show that it is possible to achieve the state of the art performance on the widely used MSCOCO benchmark.

References in corpus (6)

• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• A Deep Reinforced Model for Abstractive Summarization
• Improved Image Captioning via Policy Gradient optimization of SPIDEr
• Image Captioning with Semantic Attention
• Language Models for Image Captioning: The Quirks and What Works

Cited by in corpus (25)

• Context-Aware Visual Policy Network for Fine-Grained Image Captioning
• Context-Aware Visual Policy Network for Sequence-Level Image Captioning
• Discriminability objective for training descriptive captions
• Multi-modal gated recurrent units for image description
• Counterfactual Critic Multi-Agent Training for Scene Graph Generation
• Learning a Deep Embedding Model for Zero-Shot Learning
• ClipCap: CLIP Prefix for Image Captioning
• A Comprehensive Survey of Deep Learning for Image Captioning
• Fashion Captioning: Towards Generating Accurate Descriptions with Semantic Rewards
• Teacher-Critical Training Strategies for Image Captioning
• Fine-Grained Image Captioning with Global-Local Discriminative Objective
• Curiosity-driven Reinforcement Learning for Diverse Visual Paragraph Generation
• An Integral Projection-based Semantic Autoencoder for Zero-Shot Learning
• Label-Attention Transformer with Geometrically Coherent Objects for Image Captioning
• Gaussian Smoothen Semantic Features (GSSF) -- Exploring the Linguistic Aspects of Visual Captioning in Indian Languages (Bengali) Using MSCOCO Framework
• Consensus Graph Representation Learning for Better Grounded Image Captioning
• Intention Oriented Image Captions with Guiding Objects
• Adaptive Correlated Monte Carlo for Contextual Categorical Sequence Generation
• Diverse and Styled Image Captioning Using SVD-Based Mixture of Recurrent Experts
• Transfer Reward Learning for Policy Gradient-Based Text Generation
• Goal-driven text descriptions for images
• Learning Energy-Based Approximate Inference Networks for Structured Applications in NLP
• Do not let the history haunt you -- Mitigating Compounding Errors in Conversational Question Answering
• Depth Guided Adaptive Meta-Fusion Network for Few-shot Video Recognition
• Image Captioning using Multiple Transformers for Self-Attention Mechanism