Improved Image Captioning via Policy Gradient optimization of SPIDEr
arXiv:1612.00370 · doi:10.1109/ICCV.2017.100
Abstract
Current image captioning methods are usually trained via (penalized) maximum likelihood estimation. However, the log-likelihood score of a caption does not correlate well with human assessments of quality. Standard syntactic evaluation metrics, such as BLEU, METEOR and ROUGE, are also not well correlated. The newer SPICE and CIDEr metrics are better correlated, but have traditionally been hard to optimize for. In this paper, we show how to use a policy gradient (PG) method to directly optimize a linear combination of SPICE and CIDEr (a combination we call SPIDEr): the SPICE score ensures our captions are semantically faithful to the image, while CIDEr score ensures our captions are syntactically fluent. The PG method we propose improves on the prior MIXER approach, by using Monte Carlo rollouts instead of mixing MLE training with PG. We show empirically that our algorithm leads to easier optimization and improved results compared to MIXER. Finally, we show that using our PG method we can optimize any of the metrics, including the proposed SPIDEr metric which results in image captions that are strongly preferred by human raters compared to captions generated by the same model but trained to optimize MLE or the COCO metrics.
Accepted at ICCV 2017
References in corpus (3)
Cited by in corpus (115)
- nocaps: novel object captioning at scale
- Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generation
- Evaluation of Text Generation: A Survey
- Deep Learning Approaches on Image Captioning: A Review
- Adversarial Ranking for Language Generation
- Context-Aware Visual Policy Network for Fine-Grained Image Captioning
- A Survey of Natural Language Generation
- Bayesian Recurrent Neural Networks
- WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Research
- Context-Aware Visual Policy Network for Sequence-Level Image Captioning
- Towards Diverse and Natural Image Descriptions via a Conditional GAN
- Actor-Critic Sequence Training for Image Captioning
- Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering
- Best of Both Worlds: Transferring Knowledge from Discriminative Learning to a Generative Visual Dialog Model
- Medical-VLBERT: Medical Visual Language BERT for COVID-19 CT Report Generation With Alternate Learning
- Improving Image Captioning with Conditional Generative Adversarial Nets
- Discriminability objective for training descriptive captions
- Diverse and Accurate Image Description Using a Variational Auto-Encoder with an Additive Gaussian Encoding Space
- Towards Lightweight Transformer via Group-wise Transformation for Vision-and-Language Tasks
- Dual Attention on Pyramid Feature Maps for Image Captioning
- Trends in Integration of Vision and Language Research: A Survey of Tasks, Datasets, and Methods
- Video Storytelling: Textual Summaries for Events
- COMIC: Towards A Compact Image Captioning Model with Attention
- Automated Audio Captioning: An Overview of Recent Progress and New Challenges
- Reinforcement Learning-powered Semantic Communication via Semantic Similarity
- Visual Question Answering for Cultural Heritage
- Teaching Machines to Describe Images via Natural Language Feedback
- Learning Visual Relation Priors for Image-Text Matching and Image Captioning with Neural Scene Graph Generators
- Speaking the Same Language: Matching Machine to Human Captions by Adversarial Training
- Interactive Image Generation Using Scene Graphs
- ImageAssist: Tools for Enhancing Touchscreen-Based Image Exploration Systems for Blind and Low Vision Users
- MusCaps: Generating Captions for Music Audio
- Improving Sequence-to-Sequence Learning via Optimal Transport
- Neural Language Generation: Formulation, Methods, and Evaluation
- Say As You Wish: Fine-grained Control of Image Caption Generation with Abstract Scene Graphs
- Video Captioning via Hierarchical Reinforcement Learning
- Rethinking the Reference-based Distinctive Image Captioning
- Auxiliary Signal-Guided Knowledge Encoder-Decoder for Medical Report Generation
- Hierarchically Structured Reinforcement Learning for Topically Coherent Visual Story Generation
- An Encoder-Decoder Based Audio Captioning System With Transfer and Reinforcement Learning
- Counterfactual Critic Multi-Agent Training for Scene Graph Generation
- SemStyle: Learning to Generate Stylised Image Captions using Unaligned Text
- Consensus-based Sequence Training for Video Captioning
- Describing like humans: on diversity in image captioning
- Multi-task Regularization Based on Infrequent Classes for Audio Captioning
- Where is the Model Looking At?--Concentrate and Explain the Network Attention
- Hierarchical LSTMs with Adaptive Attention for Visual Captioning
- Asking the Difficult Questions: Goal-Oriented Visual Question Generation via Intermediate Rewards
- CL4AC: A Contrastive Loss for Audio Captioning
- Learning to Evaluate Image Captioning
- Graph Attention for Automated Audio Captioning
- VisualGPT: Data-efficient Adaptation of Pretrained Language Models for Image Captioning
- Local Information Assisted Attention-free Decoder for Audio Captioning
- Chat as Expected: Learning to Manipulate Black-box Neural Dialogue Models
- Fast, Diverse and Accurate Image Captioning Guided By Part-of-Speech
- A Comprehensive Survey of Deep Learning for Image Captioning
- From Plots to Endings: A Reinforced Pointer Generator for Story Ending Generation
- Effects of Word-frequency based Pre- and Post- Processings for Audio Captioning
- Sentiment-oriented Transformer-based Variational Autoencoder Network for Live Video Commenting
- Deep Reinforcement Learning Attention Selection for Person Re-Identification
- Unpaired Image Captioning by Language Pivoting
- Vision and Language: from Visual Perception to Content Creation
- WaveTransformer: A Novel Architecture for Audio Captioning Based on Learning Temporal and Time-Frequency Information
- LCEval: Learned Composite Metric for Caption Evaluation
- ALBA : Reinforcement Learning for Video Object Segmentation
- Gated Hierarchical Attention for Image Captioning
- Audio Captioning Using Sound Event Detection
- Discourse-Aware Neural Rewards for Coherent Text Generation
- Curiosity-driven Reinforcement Learning for Diverse Visual Paragraph Generation
- Image Captioning Based on a Hierarchical Attention Mechanism and Policy Gradient Optimization
- End-to-End Video Captioning with Multitask Reinforcement Learning
- A request for clarity over the End of Sequence token in the Self-Critical Sequence Training
- Strength in Numbers: Trading-off Robustness and Computation via Adversarially-Trained Ensembles
- Listen carefully and tell: an audio captioning system based on residual learning and gammatone audio representation
- Expressing Visual Relationships via Language
- Temporal Sub-sampling of Audio Feature Sequences for Automated Audio Captioning
- Image Captioning with Context-Aware Auxiliary Guidance
- Generating Diverse and Informative Natural Language Fashion Feedback
- Bringing back simplicity and lightliness into neural image captioning
- Evaluating Off-the-Shelf Machine Listening and Natural Language Models for Automated Audio Captioning
- Closed-Book Training to Improve Summarization Encoder Memory
- Towards Accurate Text-based Image Captioning with Content Diversity Exploration
- Gaussian Smoothen Semantic Features (GSSF) -- Exploring the Linguistic Aspects of Visual Captioning in Indian Languages (Bengali) Using MSCOCO Framework
- Towards Unique and Informative Captioning of Images
- Context-Aware Group Captioning via Self-Attention and Contrastive Features
- New Ideas and Trends in Deep Multimodal Content Understanding: A Review
- Show, Control and Tell: A Framework for Generating Controllable and Grounded Captions
- Not All Words are Equal: Video-specific Information Loss for Video Captioning
- MLE-guided parameter search for task loss minimization in neural sequence modeling
- Adaptive Correlated Monte Carlo for Contextual Categorical Sequence Generation
- DeepOpht: Medical Report Generation for Retinal Images via Deep Models and Visual Explanation
- VIFIDEL: Evaluating the Visual Fidelity of Image Descriptions
- A sequential guiding network with attention for image captioning
- Intention Oriented Image Captions with Guiding Objects
- Contextualized Keyword Representations for Multi-modal Retinal Image Captioning
- simNet: Stepwise Image-Topic Merging Network for Generating Detailed and Comprehensive Image Captions
- Mining for meaning: from vision to language through multiple networks consensus
- Learning a Reinforced Agent for Flexible Exposure Bracketing Selection
- SMURF: SeMantic and linguistic UndeRstanding Fusion for Caption Evaluation via Typicality Analysis
- Parallel Attention: A Unified Framework for Visual Object Discovery through Dialogs and Queries
- Assisting Scene Graph Generation with Self-Supervision
- Tell-the-difference: Fine-grained Visual Descriptor via a Discriminating Referee
- Nested-Wasserstein Self-Imitation Learning for Sequence Generation
- Goal-driven text descriptions for images
- Boosted Attention: Leveraging Human Attention for Image Captioning
- Learning Compact Reward for Image Captioning
- Longer Version for "Deep Context-Encoding Network for Retinal Image Captioning"
- Enhanced Modality Transition for Image Captioning
- Image Captioning using Multiple Transformers for Self-Attention Mechanism
- A Unified Framework of Surrogate Loss by Refactoring and Interpolation
- Scene-based Factored Attention for Image Captioning
- Learn to Talk via Proactive Knowledge Transfer
- PR Product: A Substitute for Inner Product in Neural Networks
- DORB: Dynamically Optimizing Multiple Rewards with Bandits
- Positioning yourself in the maze of Neural Text Generation: A Task-Agnostic Survey