DualNet: Domain-Invariant Network for Visual Question Answering
arXiv:1606.06108
Abstract
Visual question answering (VQA) task not only bridges the gap between images and language, but also requires that specific contents within the image are understood as indicated by linguistic context of the question, in order to generate the accurate answers. Thus, it is critical to build an efficient embedding of images and texts. We implement DualNet, which fully takes advantage of discriminative power of both image and textual features by separately performing two operations. Building an ensemble of DualNet further boosts the performance. Contrary to common belief, our method proved effective in both real images and abstract scenes, in spite of significantly different properties of respective domain. Our method was able to outperform previous state-of-the-art methods in real images category even without explicitly employing attention mechanism, and also outperformed our own state-of-the-art method in abstract scenes category, which recently won the first place in VQA Challenge 2016.
Accepted as an oral paper by ICME 2017
References in corpus (6)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Caffe: Convolutional Architecture for Fast Feature Embedding
- Dynamic Memory Networks for Visual and Textual Question Answering
- Multimodal Residual Learning for Visual QA
- A Focused Dynamic Attention Model for Visual Question Answering
- Dense Image Representation with Spatial Pyramid VLAD Coding of CNN for Locally Robust Captioning
Cited by in corpus (7)
- Visual Question Answering: Datasets, Algorithms, and Future Challenges
- Multi-modal Factorized Bilinear Pooling with Co-Attention Learning for Visual Question Answering
- Person Search with Natural Language Description
- Visual Question Answering: A Survey of Methods and Datasets
- An Analysis of Visual Question Answering Algorithms
- The Color of the Cat is Gray: 1 Million Full-Sentences Visual Question Answering (FSVQA)
- The VQA-Machine: Learning How to Use Existing Vision Algorithms to Answer New Questions