Visual Question Answering with Memory-Augmented Networks
arXiv:1707.04968
Abstract
In this paper, we exploit a memory-augmented neural network to predict accurate answers to visual questions, even when those answers occur rarely in the training set. The memory network incorporates both internal and external memory blocks and selectively pays attention to each training exemplar. We show that memory-augmented neural networks are able to maintain a relatively long-term memory of scarce training exemplars, which is important for visual question answering due to the heavy-tailed distribution of answers in a general VQA setting. Experimental results on two large-scale benchmark datasets show the favorable performance of the proposed algorithm with a comparison to state of the art.
CVPR 2018
References in corpus (10)
- Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
- Hierarchical Question-Image Co-Attention for Visual Question Answering
- Dynamic Memory Networks for Visual and Textual Question Answering
- Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding
- Learning to Remember Rare Events
- Multimodal Residual Learning for Visual QA
- Deep Networks with Internal Selective Attention through Feedback Connections
- Training Recurrent Answering Units with Joint Loss Minimization for VQA
- Hierarchical Memory Networks
- Recurrent Neural Networks with External Memory for Language Understanding