Cognitive Psychology for Deep Neural Networks: A Shape Bias Case Study

Abstract

Deep neural networks (DNNs) have achieved unprecedented performance on a wide range of complex tasks, rapidly outpacing our understanding of the nature of their solutions. This has caused a recent surge of interest in methods for rendering modern neural systems more interpretable. In this work, we propose to address the interpretability problem in modern DNNs using the rich history of problem descriptions, theories and experimental methods developed by cognitive psychologists to study the human mind. To explore the potential value of these tools, we chose a well-established analysis from developmental psychology that explains how children learn word labels for objects, and applied that analysis to DNNs. Using datasets of stimuli inspired by the original cognitive psychology experiments, we find that state-of-the-art one shot learning models trained on ImageNet exhibit a similar bias to that observed in humans: they prefer to categorize objects according to shape rather than color. The magnitude of this shape bias varies greatly among architecturally identical, but differently seeded models, and even fluctuates within seeds throughout training, despite nearly equivalent classification performance. These results demonstrate the capability of tools from cognitive psychology for exposing hidden computational properties of DNNs, while concurrently providing us with a computational model for human word learning.

arXiv commentICML 2017

References in corpus (9)

• Understanding Neural Networks Through Deep Visualization
• Going Deeper with Convolutions
• Matching Networks for One Shot Learning
• Visualizing and Understanding Recurrent Networks
• Model-Free Episodic Control
• Visualizing and Understanding Neural Models in NLP
• Early Visual Concept Learning with Unsupervised Deep Learning
• Towards Conceptual Compression
• Discovering objects and their relations from entangled scene representations

Cited by in corpus (8)

• Using cognitive psychology to understand GPT-3
• Psychlab: A Psychology Laboratory for Deep Reinforcement Learning Agents
• A neural network walks into a lab: towards using deep nets as models for human behavior
• Potentials and Limitations of Deep Neural Networks for Cognitive Robots
• Hierarchical Annotation of Images with Two-Alternative-Forced-Choice Metric Learning
• Understanding Deep Learning Performance through an Examination of Test Set Difficulty: A Psychometric Case Study
• Assessing The Importance Of Colours For CNNs In Object Recognition
• Towards Disentangling Latent Space for Unsupervised Semantic Face Editing