Greedy Gradient Ensemble for Robust Visual Question Answering
arXiv:2107.12651
Abstract
Language bias is a critical issue in Visual Question Answering (VQA), where models often exploit dataset biases for the final decision without considering the image information. As a result, they suffer from performance drop on out-of-distribution data and inadequate visual explanation. Based on experimental analysis for existing robust VQA methods, we stress the language bias in VQA that comes from two aspects, i.e., distribution bias and shortcut bias. We further propose a new de-bias framework, Greedy Gradient Ensemble (GGE), which combines multiple biased models for unbiased base model learning. With the greedy strategy, GGE forces the biased models to over-fit the biased data distribution in priority, thus makes the base model pay more attention to examples that are hard to solve by biased models. The experiments demonstrate that our method makes better use of visual information and achieves state-of-the-art performance on diagnosing dataset VQA-CP without using extra annotations.
arXiv commentAccepted by ICCV 2021. Code: https://github.com/GeraldHan/GGE
References in corpus (7)
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Self-Critical Reasoning for Robust Visual Question Answering
- On the Value of Out-of-Distribution Testing: An Example of Goodhart's Law
- Multi-modality Latent Interaction Network for Visual Question Answering
- Counterfactual VQA: A Cause-Effect Look at Language Bias
- Why Attentions May Not Be Interpretable?