Seeing eye-to-eye? A comparison of object recognition performance in humans and deep convolutional neural networks under image manipulation

Abstract

For a considerable time, deep convolutional neural networks (DCNNs) have reached human benchmark performance in object recognition. On that account, computational neuroscience and the field of machine learning have started to attribute numerous similarities and differences to artificial and biological vision. This study aims towards a behavioral comparison of visual core object recognition performance between humans and feedforward neural networks in a classification learning paradigm on an ImageNet data set. For this purpose, human participants (n = 65) competed in an online experiment against different feedforward DCNNs. The designed approach based on a typical learning process of seven different monkey categories included a training and validation phase with natural examples, as well as a testing phase with novel, unexperienced shape and color manipulations. Analyses of accuracy revealed that humans not only outperform DCNNs on all conditions, but also display significantly greater robustness towards shape and most notably color alterations. Furthermore, a precise examination of behavioral patterns highlights these findings by revealing independent classification errors between the groups. The obtained results show that humans contrast strongly with artificial feedforward architectures when it comes to visual core object recognition of manipulated images. In general, these findings are in line with a growing body of literature, that hints towards recurrence as a crucial factor for adequate generalization abilities.

arXiv comment19 pages, 7 figures, 3 tables

References in corpus (12)

• Intriguing properties of neural networks
• Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition
• Generalisation in humans and deep neural networks
• Superhuman Accuracy on the SNEMI3D Connectomics Challenge
• Natural Adversarial Examples
• Comparing deep neural networks against humans: object recognition when the signal gets weaker
• Task-Driven Convolutional Recurrent Models of the Visual System
• Training Spiking Deep Networks for Neuromorphic Hardware
• Going in circles is the way forward: the role of recurrence in visual inference
• Bio-Inspired Spiking Convolutional Neural Network using Layer-wise Sparse Coding and STDP Learning
• Excessive Invariance Causes Adversarial Vulnerability
• Robustly representing uncertainty in deep neural networks through sampling