Comparing deep neural networks against humans: object recognition when the signal gets weaker

Abstract

Human visual object recognition is typically rapid and seemingly effortless, as well as largely independent of viewpoint and object orientation. Until very recently, animate visual systems were the only ones capable of this remarkable computational feat. This has changed with the rise of a class of computer vision algorithms called deep neural networks (DNNs) that achieve human-level classification performance on object recognition tasks. Furthermore, a growing number of studies report similarities in the way DNNs and the human visual system process objects, suggesting that current DNNs may be good models of human visual object recognition. Yet there clearly exist important architectural and processing differences between state-of-the-art DNNs and the primate visual system. The potential behavioural consequences of these differences are not well understood. We aim to address this issue by comparing human and DNN generalisation abilities towards image degradations. We find the human visual system to be more robust to image manipulations like contrast reduction, additive noise or novel eidolon-distortions. In addition, we find progressively diverging classification error-patterns between humans and DNNs when the signal gets weaker, indicating that there may still be marked differences in the way humans and current DNNs perform visual object recognition. We envision that our findings as well as our carefully measured and freely available behavioural datasets provide a new useful benchmark for the computer vision community to improve the robustness of DNNs and a motivation for neuroscientists to search for mechanisms in the brain that could facilitate this robustness.

updated article with reference to resulting publication (Geirhos et al, NeurIPS 2018)

References in corpus (4)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• scikit-image: Image processing in Python
• Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition
• Generalisation in humans and deep neural networks

Cited by in corpus (40)

• Convolutional Neural Networks as a Model of the Visual System: Past, Present, and Future
• Neural network models and deep learning - a primer for biologists
• Generalisation in humans and deep neural networks
• Benchmarking Neural Network Robustness to Common Corruptions and Surface Variations
• GLAC Net: GLocal Attention Cascading Networks for Multi-image Cued Story Generation
• A neural network walks into a lab: towards using deep nets as models for human behavior
• Learning Loss for Test-Time Augmentation
• Distinguishing mirror from glass: A 'big data' approach to material perception
• Beyond accuracy: quantifying trial-by-trial behaviour of CNNs and humans by measuring error consistency
• CIFAR10 to Compare Visual Recognition Performance between Deep Neural Networks and Humans
• Quantifying Legibility of Indoor Spaces Using Deep Convolutional Neural Networks: Case Studies in Train Stations
• Representation Based Complexity Measures for Predicting Generalization in Deep Learning
• Solving Bongard Problems with a Visual Language and Pragmatic Reasoning
• Training with the Invisibles: Obfuscating Images to Share Safely for Learning Visual Recognition Models
• Orthogonal Deep Neural Networks
• An Overview of Perception Methods for Horticultural Robots: From Pollination to Harvest
• Robust neural circuit reconstruction from serial electron microscopy with convolutional recurrent networks
• Rearchitecting Classification Frameworks For Increased Robustness
• A fully recurrent feature extraction for single channel speech enhancement
• How is Contrast Encoded in Deep Neural Networks?
• Nonlinear Regression with a Convolutional Encoder-Decoder for Remote Monitoring of Surface Electrocardiograms
• Deep Neural Models for color discrimination and color constancy
• Deep Neural Network Based Real-time Kiwi Fruit Flower Detection in an Orchard Environment
• Ultra-low power on-chip learning of speech commands with phase-change memories
• Representation of White- and Black-Box Adversarial Examples in Deep Neural Networks and Humans: A Functional Magnetic Resonance Imaging Study
• Natural Perturbed Training for General Robustness of Neural Network Classifiers
• The Barrier of meaning in archaeological data science
• Seeing eye-to-eye? A comparison of object recognition performance in humans and deep convolutional neural networks under image manipulation
• Defective Convolutional Networks
• Seeing in the dark with recurrent convolutional neural networks
• Anomaly Detection in Video Data Based on Probabilistic Latent Space Models
• Manifestation of Image Contrast in Deep Networks
• Metamorphic Testing of a Deep Learning based Forecaster
• Top-K Deep Video Analytics: A Probabilistic Approach
• Wiggling Weights to Improve the Robustness of Classifiers
• Network-Agnostic Knowledge Transfer for Medical Image Segmentation
• Probabilistic Neural Network: Frequency and Moment Learnings
• A Deeper Look at the Unsupervised Learning of Disentangled Representations in $\beta$-VAE from the Perspective of Core Object Recognition
• Networks with pixels embedding: a method to improve noise resistance in images classification
• Sparsifying and Down-scaling Networks to Increase Robustness to Distortions