On Detecting Adversarial Perturbations

Abstract

Machine learning and deep learning in particular has advanced tremendously on perceptual tasks in recent years. However, it remains vulnerable against adversarial perturbations of the input that have been crafted specifically to fool the system while being quasi-imperceptible to a human. In this work, we propose to augment deep neural networks with a small "detector" subnetwork which is trained on the binary classification task of distinguishing genuine data from data containing adversarial perturbations. Our method is orthogonal to prior work on addressing adversarial perturbations, which has mostly focused on making the classification network itself more robust. We show empirically that adversarial perturbations can be detected surprisingly well even though they are quasi-imperceptible to humans. Moreover, while the detectors have been trained to detect only a specific adversary, they generalize to similar and weaker adversaries. In addition, we propose an adversarial attack that fools both the classifier and the detector and a novel training procedure for the detector that counteracts this attack.

Final version for ICLR2017 (see https://openreview.net/forum?id=SJzCSf9xg&noteId=SJzCSf9xg)

Cited by in corpus (92)

• Synthesizing Robust Adversarial Examples
• Understanding Adversarial Attacks on Deep Learning Based Medical Image Analysis Systems
• On the (Statistical) Detection of Adversarial Examples
• A General Framework for Adversarial Examples with Objectives
• Adversarial Example Detection for DNN Models: A Review and Experimental Comparison
• Local Model Poisoning Attacks to Byzantine-Robust Federated Learning
• Adversarial Machine Learning in Image Classification: A Survey Towards the Defender's Perspective
• Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples
• Defense Against Adversarial Attacks Using Feature Scattering-based Adversarial Training
• Benchmarking the Robustness of Semantic Segmentation Models
• Interpreting and Improving Adversarial Robustness of Deep Neural Networks with Neuron Sensitivity
• AttriGuard: A Practical Defense Against Attribute Inference Attacks via Adversarial Machine Learning
• Malware Makeover: Breaking ML-based Static Analysis by Modifying Executable Bytes
• Security and Privacy Issues in Deep Learning
• Detecting and Diagnosing Adversarial Images with Class-Conditional Capsule Reconstructions
• Adversarial Examples in Modern Machine Learning: A Review
• Defending Against Universal Attacks Through Selective Feature Regeneration
• PatchGuard: A Provably Robust Defense against Adversarial Patches via Small Receptive Fields and Masking
• Confidence-Calibrated Adversarial Training: Generalizing to Unseen Attacks
• Certified Robustness for Top-k Predictions against Adversarial Perturbations via Randomized Smoothing
• Divide, Denoise, and Defend against Adversarial Attacks
• Understanding the Decision Boundary of Deep Neural Networks: An Empirical Study
• Adversarial Learning in Statistical Classification: A Comprehensive Review of Defenses Against Attacks
• PatchCleanser: Certifiably Robust Defense against Adversarial Patches for Any Image Classifier
• Intriguing properties of adversarial training at scale
• ML-LOO: Detecting Adversarial Examples with Feature Attribution
• RAID: Randomized Adversarial-Input Detection for Neural Networks
• DaST: Data-free Substitute Training for Adversarial Attacks
• Bag of Tricks for Adversarial Training
• Noise Sensitivity-Based Energy Efficient and Robust Adversary Detection in Neural Networks
• Defending Against Physically Realizable Attacks on Image Classification
• Adversarial Example Games
• Adversarial Examples on Object Recognition: A Comprehensive Survey
• Sitatapatra: Blocking the Transfer of Adversarial Samples
• Adversarial Attacks Beyond the Image Space
• GhostImage: Remote Perception Attacks against Camera-based Image Classification Systems
• Effective and Robust Detection of Adversarial Examples via Benford-Fourier Coefficients
• Mitigating Evasion Attacks to Deep Neural Networks via Region-based Classification
• Defending against Machine Learning based Inference Attacks via Adversarial Examples: Opportunities and Challenges
• Using Depth for Pixel-Wise Detection of Adversarial Attacks in Crowd Counting
• LiBRe: A Practical Bayesian Approach to Adversarial Detection
• Adversarial Attacks and Defenses: An Interpretation Perspective
• Interpreting Adversarial Examples with Attributes
• Detecting Adversarial Examples from Sensitivity Inconsistency of Spatial-Transform Domain
• Defending against Adversarial Attack towards Deep Neural Networks via Collaborative Multi-task Training
• $n$-ML: Mitigating Adversarial Examples via Ensembles of Topologically Manipulated Classifiers
• TREATED:Towards Universal Defense against Textual Adversarial Attacks
• Stateful Detection of Black-Box Adversarial Attacks
• Joint Adversarial Training: Incorporating both Spatial and Pixel Attacks
• Cassandra: Detecting Trojaned Networks from Adversarial Perturbations
• Towards Defending Multiple ${\ell }_p$-norm Bounded Adversarial Perturbations via Gated Batch Normalization
• Almost Tight L0-norm Certified Robustness of Top-k Predictions against Adversarial Perturbations
• Adversarial Learning with Margin-based Triplet Embedding Regularization
• Exploiting the Inherent Limitation of L0 Adversarial Examples
• Automated Discovery of Adaptive Attacks on Adversarial Defenses
• BUZz: BUffer Zones for defending adversarial examples in image classification
• Transferable, Controllable, and Inconspicuous Adversarial Attacks on Person Re-identification With Deep Mis-Ranking
• Universal Adversarial Perturbations Through the Lens of Deep Steganography: Towards A Fourier Perspective
• Toward Adversarial Robustness via Semi-supervised Robust Training
• Adversarial Training Makes Weight Loss Landscape Sharper in Logistic Regression
• Towards Adversarially Robust Object Detection
• Medical Aegis: Robust adversarial protectors for medical images
• DetectorGuard: Provably Securing Object Detectors against Localized Patch Hiding Attacks
• Dompteur: Taming Audio Adversarial Examples
• Detecting and Correcting Adversarial Images Using Image Processing Operations
• ExAD: An Ensemble Approach for Explanation-based Adversarial Detection
• A Deep Learning-based Fine-grained Hierarchical Learning Approach for Robust Malware Classification
• What Do Adversarially Robust Models Look At?
• Moving Target Defense for Deep Visual Sensing against Adversarial Examples
• Detecting Adversarial Patches with Class Conditional Reconstruction Networks
• Unsupervised Detection of Adversarial Examples with Model Explanations
• What Deep CNNs Benefit from Global Covariance Pooling: An Optimization Perspective
• Immuno-mimetic Deep Neural Networks (Immuno-Net)
• A Survey on Assessing the Generalization Envelope of Deep Neural Networks: Predictive Uncertainty, Out-of-distribution and Adversarial Samples
• Perceptual Deep Neural Networks: Adversarial Robustness through Input Recreation
• A Hierarchical Feature Constraint to Camouflage Medical Adversarial Attacks
• MixDefense: A Defense-in-Depth Framework for Adversarial Example Detection Based on Statistical and Semantic Analysis
• Inconspicuous Adversarial Patches for Fooling Image Recognition Systems on Mobile Devices
• DetectX -- Adversarial Input Detection using Current Signatures in Memristive XBar Arrays
• Exploiting the Sensitivity of $L_2$ Adversarial Examples to Erase-and-Restore
• AID-Purifier: A Light Auxiliary Network for Boosting Adversarial Defense
• Deep Adversarially-Enhanced k-Nearest Neighbors
• Interpreting Attributions and Interactions of Adversarial Attacks
• Adversarial Machine Learning for Cybersecurity and Computer Vision: Current Developments and Challenges
• Ensemble of Models Trained by Key-based Transformed Images for Adversarially Robust Defense Against Black-box Attacks
• Attack-agnostic Adversarial Detection on Medical Data Using Explainable Machine Learning
• HAWKEYE: Adversarial Example Detector for Deep Neural Networks
• Decoder-free Robustness Disentanglement without (Additional) Supervision
• Adversarial Imitation Attack
• ROSA: Robust Salient Object Detection against Adversarial Attacks
• Adversarial Defense via Local Flatness Regularization
• Are L2 adversarial examples intrinsically different?