Towards Deep Neural Network Architectures Robust to Adversarial Examples

Abstract

Recent work has shown deep neural networks (DNNs) to be highly susceptible to well-designed, small perturbations at the input layer, or so-called adversarial examples. Taking images as an example, such distortions are often imperceptible, but can result in 100% mis-classification for a state of the art DNN. We study the structure of adversarial examples and explore network topology, pre-processing and training strategies to improve the robustness of DNNs. We perform various experiments to assess the removability of adversarial examples by corrupting with additional noise and pre-processing with denoising autoencoders (DAEs). We find that DAEs can remove substantial amounts of the adversarial noise. How- ever, when stacking the DAE with the original DNN, the resulting network can again be attacked by new adversarial examples with even smaller distortion. As a solution, we propose Deep Contractive Network, a model with a new end-to-end training procedure that includes a smoothness penalty inspired by the contractive autoencoder (CAE). This increases the network robustness to adversarial examples, without a significant performance penalty.

References in corpus (5)

• Going Deeper with Convolutions
• Deep Learning using Linear Support Vector Machines
• Deep Networks with Internal Selective Attention through Feedback Connections
• What Regularized Auto-Encoders Learn from the Data Generating Distribution
• PANDA: Pose Aligned Networks for Deep Attribute Modeling

Cited by in corpus (211)

• Explaining and Harnessing Adversarial Examples
• Ensemble Adversarial Training: Attacks and Defenses
• Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks
• DeepXplore: Automated Whitebox Testing of Deep Learning Systems
• Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality
• Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
• Understanding Adversarial Training: Increasing Local Stability of Neural Nets through Robust Optimization
• Optimization Problems for Machine Learning: A Survey
• Generating Adversarial Malware Examples for Black-Box Attacks Based on GAN
• Distributional Smoothing with Virtual Adversarial Training
• Adversarial Perturbations Against Deep Neural Networks for Malware Classification
• DLFuzz: Differential Fuzzing Testing of Deep Learning Systems
• Sensitivity and Generalization in Neural Networks: an Empirical Study
• PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examples
• A study of the effect of JPG compression on adversarial images
• Adversarial Examples: Opportunities and Challenges
• Keeping the Bad Guys Out: Protecting and Vaccinating Deep Learning with JPEG Compression
• Adversarial Examples: Attacks and Defenses for Deep Learning
• Spectral Norm Regularization for Improving the Generalizability of Deep Learning
• Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning
• DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Cars
• NO Need to Worry about Adversarial Examples in Object Detection in Autonomous Vehicles
• Security and Privacy Approaches in Mixed Reality: A Literature Survey
• Adversarial Weight Perturbation Helps Robust Generalization
• Measuring the tendency of CNNs to Learn Surface Statistical Regularities
• On the Convergence and Robustness of Adversarial Training
• Adversarial Example Detection for DNN Models: A Review and Experimental Comparison
• Are Labels Required for Improving Adversarial Robustness?
• Simple Black-Box Adversarial Perturbations for Deep Networks
• Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey
• Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing
• Motivating the Rules of the Game for Adversarial Example Research
• Foveation-based Mechanisms Alleviate Adversarial Examples
• How Deep Learning Sees the World: A Survey on Adversarial Attacks & Defenses
• Constructing Unrestricted Adversarial Examples with Generative Models
• Robustness of classifiers: from adversarial to random noise
• Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attack
• The Robust Manifold Defense: Adversarial Training using Generative Models
• Rademacher Complexity for Adversarially Robust Generalization
• DeepFool: a simple and accurate method to fool deep neural networks
• Adversarial Examples that Fool Detectors
• Evasion and Hardening of Tree Ensemble Classifiers
• Biologically inspired protection of deep networks from adversarial attacks
• Practical Black-Box Attacks against Machine Learning
• Blocking Transferability of Adversarial Examples in Black-Box Learning Systems
• Physics-Driven Regularization of Deep Neural Networks for Enhanced Engineering Design and Analysis
• Secure and Trustworthy Artificial Intelligence-Extended Reality (AI-XR) for Metaverses
• APE-GAN: Adversarial Perturbation Elimination with GAN
• Adversarial Example Defenses: Ensembles of Weak Defenses are not Strong
• Ensemble Methods as a Defense to Adversarial Perturbations Against Deep Neural Networks
• Exploring the Space of Black-box Attacks on Deep Neural Networks
• Adversarial Attack and Defense on Point Sets
• Towards Practical Verification of Machine Learning: The Case of Computer Vision Systems
• A Simple Explanation for the Existence of Adversarial Examples with Small Hamming Distance
• The Limitations of Deep Learning in Adversarial Settings
• Robust Convolutional Neural Networks under Adversarial Noise
• Adversarial Manipulation of Deep Representations
• Adversarial Images for Variational Autoencoders
• Improving Adversarial Robustness via Channel-wise Activation Suppressing
• Towards Evaluating the Robustness of Deep Diagnostic Models by Adversarial Attack
• Testing and verification of neural-network-based safety-critical control software: A systematic literature review
• Standard detectors aren't (currently) fooled by physical adversarial stop signs
• Detecting Adversarial Examples by Input Transformations, Defense Perturbations, and Voting
• Shield: Fast, Practical Defense and Vaccination for Deep Learning using JPEG Compression
• Defending against Adversarial Images using Basis Functions Transformations
• Understanding and Improving Fast Adversarial Training
• Adversarial Examples - A Complete Characterisation of the Phenomenon
• Detecting Adversarial Samples for Deep Neural Networks through Mutation Testing
• Robust Learning with Jacobian Regularization
• Biometric Backdoors: A Poisoning Attack Against Unsupervised Template Updating
• Denoised Smoothing: A Provable Defense for Pretrained Classifiers
• Do Wider Neural Networks Really Help Adversarial Robustness?
• Daedalus: Breaking Non-Maximum Suppression in Object Detection via Adversarial Examples
• Defend Deep Neural Networks Against Adversarial Examples via Fixed and Dynamic Quantized Activation Functions
• Adversarial Attacks and Defenses in Physiological Computing: A Systematic Review
• Fortified Networks: Improving the Robustness of Deep Networks by Modeling the Manifold of Hidden Representations
• Security Matters: A Survey on Adversarial Machine Learning
• Learning Adversary-Resistant Deep Neural Networks
• Opportunities and Challenges in Deep Learning Adversarial Robustness: A Survey
• A Theoretical Framework for Robustness of (Deep) Classifiers against Adversarial Examples
• Divide, Denoise, and Defend against Adversarial Attacks
• Are Generative Classifiers More Robust to Adversarial Attacks?
• Adversarial Examples in Deep Learning: Characterization and Divergence
• GenAttack: Practical Black-box Attacks with Gradient-Free Optimization
• A Unified Gradient Regularization Family for Adversarial Examples
• ConFoc: Content-Focus Protection Against Trojan Attacks on Neural Networks
• Uncertainty Injection: A Deep Learning Method for Robust Optimization
• Morphence: Moving Target Defense Against Adversarial Examples
• Defense against Adversarial Attacks Using High-Level Representation Guided Denoiser
• The Faults in Our Pi Stars: Security Issues and Open Challenges in Deep Reinforcement Learning
• Suppressing the Unusual: towards Robust CNNs using Symmetric Activation Functions
• Improving Back-Propagation by Adding an Adversarial Gradient
• HashTran-DNN: A Framework for Enhancing Robustness of Deep Neural Networks against Adversarial Malware Samples
• More Data Can Expand the Generalization Gap Between Adversarially Robust and Standard Models
• Brain Programming is Immune to Adversarial Attacks: Towards Accurate and Robust Image Classification using Symbolic Learning
• DUP-Net: Denoiser and Upsampler Network for 3D Adversarial Point Clouds Defense
• PeerNets: Exploiting Peer Wisdom Against Adversarial Attacks
• Unifying Adversarial Training Algorithms with Flexible Deep Data Gradient Regularization
• Improving adversarial robustness of deep neural networks by using semantic information
• Learning to Discriminate Perturbations for Blocking Adversarial Attacks in Text Classification
• Adversarial Examples Detection in Deep Networks with Convolutional Filter Statistics
• FenceBox: A Platform for Defeating Adversarial Examples with Data Augmentation Techniques
• A Provable Defense for Deep Residual Networks
• Ensemble Robustness and Generalization of Stochastic Deep Learning Algorithms
• Mitigating Advanced Adversarial Attacks with More Advanced Gradient Obfuscation Techniques
• Training Robust Deep Neural Networks via Adversarial Noise Propagation
• Security and Privacy for Artificial Intelligence: Opportunities and Challenges
• Robustifying Models Against Adversarial Attacks by Langevin Dynamics
• Towards Robust Detection of Adversarial Examples
• Online Adversarial Purification based on Self-Supervision
• Improving Network Robustness against Adversarial Attacks with Compact Convolution
• Towards Security Threats of Deep Learning Systems: A Survey
• Enhancing Gradient-based Attacks with Symbolic Intervals
• Counterexample-Guided Learning of Monotonic Neural Networks
• Adversarial Examples on Object Recognition: A Comprehensive Survey
• Retrieval-Augmented Convolutional Neural Networks for Improved Robustness against Adversarial Examples
• Towards Robust Deep Neural Networks
• Calibrated Surrogate Losses for Adversarially Robust Classification
• Using Non-invertible Data Transformations to Build Adversarial-Robust Neural Networks
• Exploiting Vulnerabilities of Deep Learning-based Energy Theft Detection in AMI through Adversarial Attacks
• Adversary Resistant Deep Neural Networks with an Application to Malware Detection
• Blind Pre-Processing: A Robust Defense Method Against Adversarial Examples
• Adversarial Attacks against Neural Networks in Audio Domain: Exploiting Principal Components
• Adversarial Perturbations Prevail in the Y-Channel of the YCbCr Color Space
• Detection based Defense against Adversarial Examples from the Steganalysis Point of View
• Image Transformation can make Neural Networks more robust against Adversarial Examples
• Neural Networks Regularization Through Class-wise Invariant Representation Learning
• Towards Adversarial-Resilient Deep Neural Networks for False Data Injection Attack Detection in Power Grids
• Generative Dynamic Patch Attack
• Mitigating Evasion Attacks to Deep Neural Networks via Region-based Classification
• Laplacian Networks: Bounding Indicator Function Smoothness for Neural Network Robustness
• Detecting Overfitting via Adversarial Examples
• Stable and expressive recurrent vision models
• Adversarial Defense of Image Classification Using a Variational Auto-Encoder
• Interpreting Adversarial Examples with Attributes
• Adversarial Robustness of Supervised Sparse Coding
• Defending against Adversarial Attack towards Deep Neural Networks via Collaborative Multi-task Training
• Built-in Vulnerabilities to Imperceptible Adversarial Perturbations
• Improving Global Adversarial Robustness Generalization With Adversarially Trained GAN
• Evaluating Robustness of Predictive Uncertainty Estimation: Are Dirichlet-based Models Reliable?
• Defense-VAE: A Fast and Accurate Defense against Adversarial Attacks
• Random Directional Attack for Fooling Deep Neural Networks
• Style transfer-based image synthesis as an efficient regularization technique in deep learning
• Machine vs Machine: Minimax-Optimal Defense Against Adversarial Examples
• Toward Adversarial Robustness via Semi-supervised Robust Training
• Predicting Adversarial Examples with High Confidence
• ATRO: Adversarial Training with a Rejection Option
• Why Accuracy Is Not Enough: The Need for Consistency in Object Detection
• Technologies for Trustworthy Machine Learning: A Survey in a Socio-Technical Context
• Statistically Robust Neural Network Classification
• Provable robustness against all adversarial $l_p$-perturbations for $p\ge 1$
• Detecting Adversarial Perturbations with Saliency
• Ptolemy: Architecture Support for Robust Deep Learning
• Analytical Moment Regularizer for Gaussian Robust Networks
• Butterfly Effect: Bidirectional Control of Classification Performance by Small Additive Perturbation
• DAFAR: Defending against Adversaries by Feedback-Autoencoder Reconstruction
• FineFool: Fine Object Contour Attack via Attention
• Adaptive Neighbourhoods for the Discovery of Adversarial Examples
• Understanding the Error in Evaluating Adversarial Robustness
• Once a MAN: Towards Multi-Target Attack via Learning Multi-Target Adversarial Network Once
• Universal Approximation with Certified Networks
• Automated Detection System for Adversarial Examples with High-Frequency Noises Sieve
• Training Artificial Neural Networks by Generalized Likelihood Ratio Method: Exploring Brain-like Learning to Improve Robustness
• Deep Learning Defenses Against Adversarial Examples for Dynamic Risk Assessment
• Accelerating Training of Deep Neural Networks with a Standardization Loss
• Iterative Window Mean Filter: Thwarting Diffusion-based Adversarial Purification
• Neuron with Steady Response Leads to Better Generalization
• A Computationally Efficient Method for Defending Adversarial Deep Learning Attacks
• Adequacy of the Gradient-Descent Method for Classifier Evasion Attacks
• A New Angle on L2 Regularization
• ExAD: An Ensemble Approach for Explanation-based Adversarial Detection
• Search Space of Adversarial Perturbations against Image Filters
• The Outer Product Structure of Neural Network Derivatives
• Clipping free attacks against artificial neural networks
• Enhanced countering adversarial attacks via input denoising and feature restoring
• Augmenting Model Robustness with Transformation-Invariant Attacks
• Towards Robust Pattern Recognition: A Review
• Downscaling Attack and Defense: Turning What You See Back Into What You Get
• Regularized Ensembles and Transferability in Adversarial Learning
• Mitigation of Adversarial Attacks through Embedded Feature Selection
• An Empirical Investigation of Randomized Defenses against Adversarial Attacks
• BO-DBA: Query-Efficient Decision-Based Adversarial Attacks via Bayesian Optimization
• SPADE: A Spectral Method for Black-Box Adversarial Robustness Evaluation
• Information Security and Privacy in the Digital World: Some Selected Topics
• Weighted Average Precision: Adversarial Example Detection in the Visual Perception of Autonomous Vehicles
• Architectural Resilience to Foreground-and-Background Adversarial Noise
• Robust lEarned Shrinkage-Thresholding (REST): Robust unrolling for sparse recover
• Manifold Regularized Discriminative Neural Networks
• The Artificial Mind's Eye: Resisting Adversarials for Convolutional Neural Networks using Internal Projection
• Deep Learning-Based Autonomous Driving Systems: A Survey of Attacks and Defenses
• Training Machine Learning Models by Regularizing their Explanations
• Stochastic-HMDs: Adversarial Resilient Hardware Malware Detectors through Voltage Over-scaling
• A Training-based Identification Approach to VIN Adversarial Examples
• Attribution of Gradient Based Adversarial Attacks for Reverse Engineering of Deceptions
• On Configurable Defense against Adversarial Example Attacks
• Analyzing Adversarial Robustness of Deep Neural Networks in Pixel Space: a Semantic Perspective
• Orthogonal Deep Models As Defense Against Black-Box Attacks
• Robust, Deep, and Reinforcement Learning for Management of Communication and Power Networks
• Denoising and Verification Cross-Layer Ensemble Against Black-box Adversarial Attacks
• Improving Resistance to Adversarial Deformations by Regularizing Gradients
• Adversarial Defenses via Vector Quantization
• Likelihood Landscapes: A Unifying Principle Behind Many Adversarial Defenses
• Towards Model-Agnostic Adversarial Defenses using Adversarially Trained Autoencoders
• Learning to Disentangle Robust and Vulnerable Features for Adversarial Detection
• On educating machines
• Adversarial Data Encryption
• HAWKEYE: Adversarial Example Detector for Deep Neural Networks
• Mitigating Class Boundary Label Uncertainty to Reduce Both Model Bias and Variance
• Gödel's Sentence Is An Adversarial Example But Unsolvable
• Physical world assistive signals for deep neural network classifiers -- neither defense nor attack
• LG-GAN: Label Guided Adversarial Network for Flexible Targeted Attack of Point Cloud-based Deep Networks