TallyQA: Answering Complex Counting Questions

Abstract

Most counting questions in visual question answering (VQA) datasets are simple and require no more than object detection. Here, we study algorithms for complex counting questions that involve relationships between objects, attribute identification, reasoning, and more. To do this, we created TallyQA, the world's largest dataset for open-ended counting. We propose a new algorithm for counting that uses relation networks with region proposals. Our method lets relation networks be efficiently used with high-resolution imagery. It yields state-of-the-art results compared to baseline and recent systems on both TallyQA and the HowMany-QA benchmark.

arXiv commentTo appear in AAAI 2019 ( To download the dataset please go to http://www.manojacharya.com/ )

References in corpus (11)

• Hierarchical Question-Image Co-Attention for Visual Question Answering
• VQA: Visual Question Answering
• A simple neural network module for relational reasoning
• Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding
• Visual Question Answering: Datasets, Algorithms, and Future Challenges
• Learning to Count Objects in Natural Images for Visual Question Answering
• Learning to Compose Neural Networks for Question Answering
• Bilinear Attention Networks
• Interpretable Counting for Visual Question Answering
• DVQA: Understanding Data Visualizations via Question Answering
• An Analysis of Visual Question Answering Algorithms