Zero-Shot Transfer VQA Dataset
arXiv:1811.00692
Abstract
Acquiring a large vocabulary is an important aspect of human intelligence. Onecommon approach for human to populating vocabulary is to learn words duringreading or listening, and then use them in writing or speaking. This ability totransfer from input to output is natural for human, but it is difficult for machines.Human spontaneously performs this knowledge transfer in complicated multimodaltasks, such as Visual Question Answering (VQA). In order to approach human-levelArtificial Intelligence, we hope to equip machines with such ability. Therefore, toaccelerate this research, we propose a newzero-shot transfer VQA(ZST-VQA)dataset by reorganizing the existing VQA v1.0 dataset in the way that duringtraining, some words appear only in one module (i.e. questions) but not in theother (i.e. answers). In this setting, an intelligent model should understand andlearn the concepts from one module (i.e. questions), and at test time, transfer themto the other (i.e. predict the concepts as answers). We conduct evaluation on thisnew dataset using three existing state-of-the-art VQA neural models. Experimentalresults show a significant drop in performance on this dataset, indicating existingmethods do not address the zero-shot transfer problem. Besides, our analysis findsthat this may be caused by the implicit bias learned during training.
References in corpus (14)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Hierarchical Question-Image Co-Attention for Visual Question Answering
- Dynamic Memory Networks for Visual and Textual Question Answering
- A simple neural network module for relational reasoning
- Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding
- ABC-CNN: An Attention Based Convolutional Neural Network for Visual Question Answering
- Visual Question Answering: Datasets, Algorithms, and Future Challenges
- Compositional Attention Networks for Machine Reasoning
- Learning to Reason: End-to-End Module Networks for Visual Question Answering
- Zero-Shot Visual Question Answering
- C-VQA: A Compositional Split of the Visual Question Answering (VQA) v1.0 Dataset
- Dual Attention Networks for Multimodal Reasoning and Matching
- Survey of Visual Question Answering: Datasets and Techniques
- An Empirical Evaluation of Visual Question Answering for Novel Objects