Towards a Visual Turing Challenge
arXiv:1410.8027
Abstract
As language and visual understanding by machines progresses rapidly, we are observing an increasing interest in holistic architectures that tightly interlink both modalities in a joint learning and inference process. This trend has allowed the community to progress towards more challenging and open tasks and refueled the hope at achieving the old AI dream of building machines that could pass a turing test in open domains. In order to steadily make progress towards this goal, we realize that quantifying performance becomes increasingly difficult. Therefore we ask how we can precisely define such challenges and how we can evaluate different algorithms on this open tasks? In this paper, we summarize and discuss such challenges as well as try to give answers where appropriate options are available in the literature. We exemplify some of the solutions on a recently presented dataset of question-answering task based on real-world indoor images that establishes a visual turing challenge. Finally, we argue despite the success of unique ground-truth annotation, we likely have to step away from carefully curated dataset and rather rely on 'social consensus' as the main driving force to create suitable benchmarks. Providing coverage in this inherently ambiguous output space is an emerging challenge that we face in order to make quantifiable progress in this area.
Published in the NIPS 2014 Workshop on Learning Semantics
References in corpus (7)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
- Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
- Deeply-Supervised Nets
- Deep Fragment Embeddings for Bidirectional Image Sentence Mapping
- Coherent Multi-Sentence Video Description with Variable Level of Detail
- A Joint Model of Language and Perception for Grounded Attribute Learning
Cited by in corpus (4)
- Exploring Models and Data for Image Question Answering
- Visual Question Answering: Datasets, Algorithms, and Future Challenges
- From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunities
- Compact Trilinear Interaction for Visual Question Answering