Representation of White- and Black-Box Adversarial Examples in Deep Neural Networks and Humans: A Functional Magnetic Resonance Imaging Study
arXiv:1905.02422
Abstract
The recent success of brain-inspired deep neural networks (DNNs) in solving complex, high-level visual tasks has led to rising expectations for their potential to match the human visual system. However, DNNs exhibit idiosyncrasies that suggest their visual representation and processing might be substantially different from human vision. One limitation of DNNs is that they are vulnerable to adversarial examples, input images on which subtle, carefully designed noises are added to fool a machine classifier. The robustness of the human visual system against adversarial examples is potentially of great importance as it could uncover a key mechanistic feature that machine vision is yet to incorporate. In this study, we compare the visual representations of white- and black-box adversarial examples in DNNs and humans by leveraging functional magnetic resonance imaging (fMRI). We find a small but significant difference in representation patterns for different (i.e. white- versus black- box) types of adversarial examples for both humans and DNNs. However, human performance on categorical judgment is not degraded by noise regardless of the type unlike DNN. These results suggest that adversarial examples may be differentially represented in the human visual system, but unable to affect the perceptual experience.
arXiv commentCopyright 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works
References in corpus (8)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Explaining and Harnessing Adversarial Examples
- Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
- Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition
- Synthesizing Robust Adversarial Examples
- Adversarial Examples that Fool both Computer Vision and Time-Limited Humans
- Comparing deep neural networks against humans: object recognition when the signal gets weaker
- Modeling Human Categorization of Natural Images Using Deep Feature Representations