Visual Reasoning in Object-Centric Deep Neural Networks: A Comparative Cognition Approach

Abstract

Achieving visual reasoning is a long-term goal of artificial intelligence. In the last decade, several studies have applied deep neural networks (DNNs) to the task of learning visual relations from images, with modest results in terms of generalization of the relations learned. However, in recent years, object-centric representation learning has been put forward as a way to achieve visual reasoning within the deep learning framework. Object-centric models attempt to model input scenes as compositions of objects and relations between them. To this end, these models use several kinds of attention mechanisms to segregate the individual objects in a scene from the background and from other objects. In this work we tested relation learning and generalization in several object-centric models, as well as a ResNet-50 baseline. In contrast to previous research, which has focused heavily in the same-different task in order to asses relational reasoning in DNNs, we use a set of tasks -- with varying degrees of difficulty -- derived from the comparative cognition literature. Our results show that object-centric models are able to segregate the different objects in a scene, even in many out-of-distribution cases. In our simpler tasks, this improves their capacity to learn and generalize visual relations in comparison to the ResNet-50 baseline. However, object-centric models still struggle in our more difficult tasks and conditions. We conclude that abstract visual reasoning remains an open challenge for DNNs, including object-centric models.

arXiv comment16 pages, 14 figures

References in corpus (17)

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• DINOv2: Learning Robust Visual Features without Supervision
• Object-Centric Learning with Slot Attention
• Five Points to Check when Comparing Visual Perception in Humans and Machines
• Spatial Broadcast Decoder: A Simple Architecture for Learning Disentangled Representations in VAEs
• Human-like generalization in a machine through predicate learning
• A Theory of Relation Learning and Cross-domain Generalization
• A Review of Emerging Research Directions in Abstract Visual Reasoning
• Solving the Same-Different Task with Convolutional Neural Networks
• Understanding the computational demands underlying visual reasoning
• Using Counterfactual Tasks to Evaluate the Generality of Analogical Reasoning in Large Language Models
• On Neural Architecture Inductive Biases for Relational Tasks
• Complex-Valued Autoencoders for Object Discovery
• Systematic Visual Reasoning through Object-Centric Relational Abstraction
• Affinity-based Attention in Self-supervised Transformers Predicts Dynamics of Object Grouping in Humans
• Deep Neural Networks Can Learn Generalizable Same-Different Visual Relations
• Contrastive Training of Complex-Valued Autoencoders for Object Discovery