Biologically inspired protection of deep networks from adversarial attacks
arXiv:1703.09202
Abstract
Inspired by biophysical principles underlying nonlinear dendritic computation in neural circuits, we develop a scheme to train deep neural networks to make them robust to adversarial attacks. Our scheme generates highly nonlinear, saturated neural networks that achieve state of the art performance on gradient based adversarial examples on MNIST, despite never being exposed to adversarially chosen examples during training. Moreover, these networks exhibit unprecedented robustness to targeted, iterative schemes for generating adversarial examples, including second-order methods. We further identify principles governing how these networks achieve their robustness, drawing on methods from information geometry. We find these networks progressively create highly flat and compressed internal representations that are sensitive to very few input dimensions, while still solving the task. Moreover, they employ highly kurtotic weight distributions, also found in the brain, and we demonstrate how such kurtosis can protect even linear classifiers from adversarial attack.
arXiv comment11 pages
References in corpus (5)
Cited by in corpus (40)
- Ensemble Adversarial Training: Attacks and Defenses
- Certified Defenses against Adversarial Examples
- PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examples
- Stochastic Activation Pruning for Robust Adversarial Defense
- Adversarial Example Detection for DNN Models: A Review and Experimental Comparison
- Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey
- Motivating the Rules of the Game for Adversarial Example Research
- Adversarial Training against Location-Optimized Adversarial Patches
- Adversarial Examples - A Complete Characterisation of the Phenomenon
- Generalized Capsule Networks with Trainable Routing Procedure
- Towards a Robust Deep Neural Network in Texts: A Survey
- The Threat of Adversarial Attacks on Machine Learning in Network Security -- A Survey
- Intriguing Properties of Adversarial Examples
- Comment on "Biologically inspired protection of deep networks from adversarial attacks"
- Defense against Adversarial Attacks Using High-Level Representation Guided Denoiser
- Improving adversarial robustness of deep neural networks by using semantic information
- Analysis of Deep Networks for Monocular Depth Estimation Through Adversarial Attacks with Proposal of a Defense Method
- Adversarial Examples on Object Recognition: A Comprehensive Survey
- Deep pNML: Predictive Normalized Maximum Likelihood for Deep Neural Networks
- Computing a human-like reaction time metric from stable recurrent vision models
- Image Transformation can make Neural Networks more robust against Adversarial Examples
- Defense-VAE: A Fast and Accurate Defense against Adversarial Attacks
- Toward Learning Human-aligned Cross-domain Robust Models by Countering Misaligned Features
- Adversarial Defense by Stratified Convolutional Sparse Coding
- Defending Adversarial Attacks by Correcting logits
- Minimax Defense against Gradient-based Adversarial Attacks
- A Unified Game-Theoretic Interpretation of Adversarial Robustness
- FineFool: Fine Object Contour Attack via Attention
- Automated Detection System for Adversarial Examples with High-Frequency Noises Sieve
- Bio-inspired Robustness: A Review
- Blurred Images Lead to Bad Local Minima
- Evaluating adversarial robustness in simulated cerebellum
- And/or trade-off in artificial neurons: impact on adversarial robustness
- HAWKEYE: Adversarial Example Detector for Deep Neural Networks
- Towards Natural Robustness Against Adversarial Examples
- 3D-OOCS: Learning Prostate Segmentation with Inductive Bias
- Stochastic-HMDs: Adversarial Resilient Hardware Malware Detectors through Voltage Over-scaling
- Training Machine Learning Models by Regularizing their Explanations
- Extreme Value Preserving Networks
- Linking average- and worst-case perturbation robustness via class selectivity and dimensionality