Capturing human categorization of natural images at scale by combining deep networks and cognitive models

Abstract

Human categorization is one of the most important and successful targets of cognitive modeling in psychology, yet decades of development and assessment of competing models have been contingent on small sets of simple, artificial experimental stimuli. Here we extend this modeling paradigm to the domain of natural images, revealing the crucial role that stimulus representation plays in categorization and its implications for conclusions about how people form categories. Applying psychological models of categorization to natural images required two significant advances. First, we conducted the first large-scale experimental study of human categorization, involving over 500,000 human categorization judgments of 10,000 natural images from ten non-overlapping object categories. Second, we addressed the traditional bottleneck of representing high-dimensional images in cognitive models by exploring the best of current supervised and unsupervised deep and shallow machine learning methods. We find that selecting sufficiently expressive, data-driven representations is crucial to capturing human categorization, and using these representations allows simple models that represent categories with abstract prototypes to outperform the more complex memory-based exemplar accounts of categorization that have dominated in studies using less naturalistic stimuli.

arXiv comment29 pages; 4 figures. arXiv admin note: text overlap with arXiv:1711.04855

Cited by in corpus (8)

• Controversial stimuli: pitting neural networks against each other as models of human recognition
• Active label cleaning for improved dataset quality under resource constraints
• CLIP in Medical Imaging: A Survey
• Automating psychological hypothesis generation with AI: when large language models meet causal graph
• Human-like object concept representations emerge naturally in multimodal large language models
• Angular Gap: Reducing the Uncertainty of Image Difficulty through Model Calibration
• Integrating Semantic Communication and Human Decision-Making into an End-to-End Sensing-Decision Framework
• Exploring internal representation of self-supervised networks: few-shot learning abilities and comparison with human semantics and recognition of objects