Counterfactual Critic Multi-Agent Training for Scene Graph Generation
arXiv:1812.02347
Abstract
Scene graphs -- objects as nodes and visual relationships as edges -- describe the whereabouts and interactions of the things and stuff in an image for comprehensive scene understanding. To generate coherent scene graphs, almost all existing methods exploit the fruitful visual context by modeling message passing among objects, fitting the dynamic nature of reasoning with visual context, eg, "person" on "bike" can help to determine the relationship "ride", which in turn contributes to the category confidence of the two objects. However, we argue that the scene dynamics is not properly learned by using the prevailing cross-entropy based supervised learning paradigm, which is not sensitive to graph inconsistency: errors at the hub or non-hub nodes are unfortunately penalized equally. To this end, we propose a Counterfactual critic Multi-Agent Training (CMAT) approach to resolve the mismatch. CMAT is a multi-agent policy gradient method that frames objects as cooperative agents, and then directly maximizes a graph-level metric as the reward. In particular, to assign the reward properly to each agent, CMAT uses a counterfactual baseline that disentangles the agent-specific reward by fixing the dynamics of other agents. Extensive validations on the challenging Visual Genome benchmark show that CMAT achieves a state-of-the-art by significant performance gains under various settings and metrics.
arXiv commentInternational Conference on Computer Vision (ICCV), 2019 (oral)
References in corpus (20)
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
- Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
- Conditional Random Fields as Recurrent Neural Networks
- Learning to Communicate with Deep Multi-Agent Reinforcement Learning
- DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
- Improved Image Captioning via Policy Gradient optimization of SPIDEr
- Counterfactual Multi-Agent Policy Gradients
- Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning
- Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding
- Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
- Pixels to Graphs by Associative Embedding
- Learning to Count Objects in Natural Images for Visual Question Answering
- The Optimal Reward Baseline for Gradient-Based Reinforcement Learning
- Learning Conditioned Graph Structures for Interpretable Visual Question Answering
- Context-Aware Visual Policy Network for Sequence-Level Image Captioning
- Mapping Images to Scene Graphs with Permutation-Invariant Structured Prediction
- Actor-Critic Sequence Training for Image Captioning
- Tree-Structured Reinforcement Learning for Sequential Object Localization
- Image Generation from Scene Graphs