RAVEN: A Dataset for Relational and Analogical Visual rEasoNing

Abstract

Dramatic progress has been witnessed in basic vision tasks involving low-level perception, such as object recognition, detection, and tracking. Unfortunately, there is still an enormous performance gap between artificial vision systems and human intelligence in terms of higher-level vision problems, especially ones involving reasoning. Earlier attempts in equipping machines with high-level reasoning have hovered around Visual Question Answering (VQA), one typical task associating vision and language understanding. In this work, we propose a new dataset, built in the context of Raven's Progressive Matrices (RPM) and aimed at lifting machine intelligence by associating vision with structural, relational, and analogical reasoning in a hierarchical representation. Unlike previous works in measuring abstract reasoning using RPM, we establish a semantic link between vision and reasoning by providing structure representation. This addition enables a new type of abstract reasoning by jointly operating on the structure representation. Machine reasoning ability using modern computer vision is evaluated in this newly proposed dataset. Additionally, we also provide human performance as a reference. Finally, we show consistent improvement across all models by incorporating a simple neural module that combines visual understanding and structure reasoning.

arXiv commentCVPR 2019 paper. Supplementary: http://wellyzhang.github.io/attach/cvpr19zhang_supp.pdf Project: http://wellyzhang.github.io/project/raven.html

References in corpus (11)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Distributed Representations of Words and Phrases and their Compositionality
• Semi-Supervised Classification with Graph Convolutional Networks
• VQA: Visual Question Answering
• Exploring Models and Data for Image Question Answering
• Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding
• Transparency by Design: Closing the Gap Between Performance and Interpretability in Visual Reasoning
• Measuring abstract reasoning in neural networks
• Compositional Attention Networks for Machine Reasoning
• CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning
• IQ of Neural Networks

Cited by in corpus (9)

• Dark, Beyond Deep: A Paradigm Shift to Cognitive AI with Humanlike Common Sense
• Trends in Integration of Vision and Language Research: A Survey of Tasks, Datasets, and Methods
• Learning Visual Abstract Reasoning through Dual-Stream Networks
• Attention on Abstract Visual Reasoning
• One Self-Configurable Model to Solve Many Abstract Visual Reasoning Problems
• KANDINSKYPatterns -- An experimental exploration environment for Pattern Analysis and Machine Intelligence
• Theory-based Causal Transfer: Integrating Instance-level Induction and Abstract-level Structure Learning
• Toward Building Science Discovery Machines
• Machine Number Sense: A Dataset of Visual Arithmetic Problems for Abstract and Relational Reasoning