MemexQA: Visual Memex Question Answering
arXiv:1708.01336
Abstract
This paper proposes a new task, MemexQA: given a collection of photos or videos from a user, the goal is to automatically answer questions that help users recover their memory about events captured in the collection. Towards solving the task, we 1) present the MemexQA dataset, a large, realistic multimodal dataset consisting of real personal photos and crowd-sourced questions/answers, 2) propose MemexNet, a unified, end-to-end trainable network architecture for image, text and video question answering. Experimental results on the MemexQA dataset demonstrate that MemexNet outperforms strong baselines and yields the state-of-the-art on this novel and challenging task. The promising results on TextQA and VideoQA suggest MemexNet's efficacy and scalability across various QA tasks.
https://memexqa.cs.cmu.edu/
References in corpus (15)
- Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
- YFCC100M: The New Data in Multimedia Research
- VQA: Visual Question Answering
- SQuAD: 100,000+ Questions for Machine Comprehension of Text
- Dynamic Memory Networks for Visual and Textual Question Answering
- Dynamic Coattention Networks For Question Answering
- Machine Comprehension Using Match-LSTM and Answer Pointer
- Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding
- ReasoNet: Learning to Stop Reading in Machine Comprehension
- Ask Your Neurons: A Neural-based Approach to Answering Questions about Images
- Multi-Perspective Context Matching for Machine Comprehension
- Visual Storytelling
- Knowing When to Look: Adaptive Attention via A Visual Sentinel for Image Captioning
- Hierarchical Memory Networks
- FVQA: Fact-based Visual Question Answering
Cited by in corpus (4)
- Focal Visual-Text Attention for Visual Question Answering
- Progressive Attention Memory Network for Movie Story Question Answering
- Technical Report of the Video Event Reconstruction and Analysis (VERA) System -- Shooter Localization, Models, Interface, and Beyond
- Holistic Multi-modal Memory Network for Movie Question Answering