Answer Them All! Toward Universal Visual Question Answering Models
arXiv:1903.00366
Abstract
Visual Question Answering (VQA) research is split into two camps: the first focuses on VQA datasets that require natural image understanding and the second focuses on synthetic datasets that test reasoning. A good VQA algorithm should be capable of both, but only a few VQA algorithms are tested in this manner. We compare five state-of-the-art VQA algorithms across eight VQA datasets covering both domains. To make the comparison fair, all of the models are standardized as much as possible, e.g., they use the same visual features, answer vocabularies, etc. We find that methods do not generalize across the two domains. To address this problem, we propose a new VQA algorithm that rivals or exceeds the state-of-the-art for both domains.
8 pages
References in corpus (12)
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- VQA: Visual Question Answering
- Visual Question Answering: Datasets, Algorithms, and Future Challenges
- A Multi-World Approach to Question Answering about Real-World Scenes based on Uncertain Input
- Transparency by Design: Closing the Gap Between Performance and Interpretability in Visual Reasoning
- Measuring Catastrophic Forgetting in Neural Networks
- Hadamard Product for Low-rank Bilinear Pooling
- Learning Conditioned Graph Structures for Interpretable Visual Question Answering
- Training Recurrent Answering Units with Joint Loss Minimization for VQA
- C-VQA: A Compositional Split of the Visual Question Answering (VQA) v1.0 Dataset
- The Visual QA Devil in the Details: The Impact of Early Fusion and Batch Norm on CLEVR
- VQD: Visual Query Detection in Natural Scenes