Detecting Adversarial Samples from Artifacts

Abstract

Deep neural networks (DNNs) are powerful nonlinear architectures that are known to be robust to random perturbations of the input. However, these models are vulnerable to adversarial perturbations--small input changes crafted explicitly to fool the model. In this paper, we ask whether a DNN can distinguish adversarial samples from their normal and noisy counterparts. We investigate model confidence on adversarial samples by looking at Bayesian uncertainty estimates, available in dropout neural networks, and by performing density estimation in the subspace of deep features learned by the model. The result is a method for implicit adversarial detection that is oblivious to the attack algorithm. We evaluate this method on a variety of standard datasets including MNIST and CIFAR-10 and show that it generalizes well across different architectures and attacks. Our findings report that 85-93% ROC-AUC can be achieved on a number of standard classification tasks with a negative class that consists of both normal and noisy samples.

Submitted to ICML 2017

References in corpus (2)

• Better Mixing via Deep Representations
• A Boundary Tilting Persepective on the Phenomenon of Adversarial Examples

Cited by in corpus (187)

• Understanding Adversarial Attacks on Deep Learning Based Medical Image Analysis Systems
• Fast is better than free: Revisiting adversarial training
• On the (Statistical) Detection of Adversarial Examples
• Adversarial Examples: Opportunities and Challenges
• Keeping the Bad Guys Out: Protecting and Vaccinating Deep Learning with JPEG Compression
• A General Framework for Adversarial Examples with Objectives
• Adversarial Example Detection for DNN Models: A Review and Experimental Comparison
• Understanding Measures of Uncertainty for Adversarial Example Detection
• Motivating the Rules of the Game for Adversarial Example Research
• Adversarial Sample Detection for Deep Neural Network through Model Mutation Testing
• advertorch v0.1: An Adversarial Robustness Toolbox based on PyTorch
• Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples
• A Review of Adversarial Attack and Defense for Classification Methods
• Extending Defensive Distillation
• Ensemble Methods as a Defense to Adversarial Perturbations Against Deep Neural Networks
• A Simple Explanation for the Existence of Adversarial Examples with Small Hamming Distance
• Malware Makeover: Breaking ML-based Static Analysis by Modifying Executable Bytes
• Security and Privacy Issues in Deep Learning
• DeepSafe: A Data-driven Approach for Checking Adversarial Robustness in Neural Networks
• Towards Evaluating the Robustness of Deep Diagnostic Models by Adversarial Attack
• Detection of Face Recognition Adversarial Attacks
• Testing and verification of neural-network-based safety-critical control software: A systematic literature review
• Detecting and Diagnosing Adversarial Images with Class-Conditional Capsule Reconstructions
• Adversarial Examples in Modern Machine Learning: A Review
• Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
• Fawkes: Protecting Privacy against Unauthorized Deep Learning Models
• Overfitting in adversarially robust deep learning
• Adversarial Attacks on Time-Series Intrusion Detection for Industrial Control Systems
• Adversarial Examples - A Complete Characterisation of the Phenomenon
• A Survey on Open Set Recognition
• Why is the Mahalanobis Distance Effective for Anomaly Detection?
• Defense Methods Against Adversarial Examples for Recurrent Neural Networks
• A Causal View on Robustness of Neural Networks
• Deep Verifier Networks: Verification of Deep Discriminative Models with Deep Generative Models
• Detecting Adversarial Attacks on Neural Network Policies with Visual Foresight
• PRADA: Protecting against DNN Model Stealing Attacks
• Striking the Right Balance with Uncertainty
• Adversarial Robustness via Fisher-Rao Regularization
• Sufficient Conditions for Idealised Models to Have No Adversarial Examples: a Theoretical and Empirical Study with Bayesian Neural Networks
• Robustness of Bayesian Neural Networks to Gradient-Based Attacks
• Divide, Denoise, and Defend against Adversarial Attacks
• Are Generative Classifiers More Robust to Adversarial Attacks?
• The Threat of Adversarial Attacks on Machine Learning in Network Security -- A Survey
• Improving black-box optimization in VAE latent space using decoder uncertainty
• Uncertainty Based Detection and Relabeling of Noisy Image Labels
• Adversarial Learning in Statistical Classification: A Comprehensive Review of Defenses Against Attacks
• The Faults in Our Pi Stars: Security Issues and Open Challenges in Deep Reinforcement Learning
• When Explainability Meets Adversarial Learning: Detecting Adversarial Examples using SHAP Signatures
• CapsAttacks: Robust and Imperceptible Adversarial Attacks on Capsule Networks
• Attacking Optical Character Recognition (OCR) Systems with Adversarial Watermarks
• A Real-time Defense against Website Fingerprinting Attacks
• Intriguing properties of adversarial training at scale
• Deflecting Adversarial Attacks
• ML-LOO: Detecting Adversarial Examples with Feature Attribution
• A Provable Defense for Deep Residual Networks
• Breaking Transferability of Adversarial Samples with Randomness
• Natural and Adversarial Error Detection using Invariance to Image Transformations
• RAID: Randomized Adversarial-Input Detection for Neural Networks
• Security and Privacy for Artificial Intelligence: Opportunities and Challenges
• DaST: Data-free Substitute Training for Adversarial Attacks
• Two Souls in an Adversarial Image: Towards Universal Adversarial Example Detection using Multi-view Inconsistency
• Bag of Tricks for Adversarial Training
• A Research Agenda: Dynamic Models to Defend Against Correlated Attacks
• Improving Network Robustness against Adversarial Attacks with Compact Convolution
• AdvSPADE: Realistic Unrestricted Attacks for Semantic Segmentation
• Noise Sensitivity-Based Energy Efficient and Robust Adversary Detection in Neural Networks
• Detecting Adversarial Examples Is (Nearly) As Hard As Classifying Them
• Detecting Adversarial Examples through Nonlinear Dimensionality Reduction
• Adversarial Examples on Object Recognition: A Comprehensive Survey
• ATHENA: A Framework based on Diverse Weak Defenses for Building Adversarial Defense
• Identifying Audio Adversarial Examples via Anomalous Pattern Detection
• The Taboo Trap: Behavioural Detection of Adversarial Samples
• Detecting Adversarial Examples in Convolutional Neural Networks
• Input Validation for Neural Networks via Runtime Local Robustness Verification
• Anomalous Example Detection in Deep Learning: A Survey
• Adversarial Robustness Guarantees for Classification with Gaussian Processes
• Effective and Robust Detection of Adversarial Examples via Benford-Fourier Coefficients
• Test Selection for Deep Learning Systems
• Controlling Over-generalization and its Effect on Adversarial Examples Generation and Detection
• Detecting Adversarial Examples via Neural Fingerprinting
• Using Depth for Pixel-Wise Detection of Adversarial Attacks in Crowd Counting
• Mitigating Evasion Attacks to Deep Neural Networks via Region-based Classification
• Adversarial Attacks and Defenses: An Interpretation Perspective
• APRICOT: A Dataset of Physical Adversarial Attacks on Object Detection
• LiBRe: A Practical Bayesian Approach to Adversarial Detection
• ReabsNet: Detecting and Revising Adversarial Examples
• Attacking and Defending Machine Learning Applications of Public Cloud
• Detecting Adversarial Examples from Sensitivity Inconsistency of Spatial-Transform Domain
• Revealing Perceptible Backdoors, without the Training Set, via the Maximum Achievable Misclassification Fraction Statistic
• On the Need for Topology-Aware Generative Models for Manifold-Based Defenses
• Built-in Vulnerabilities to Imperceptible Adversarial Perturbations
• Defending against Adversarial Attack towards Deep Neural Networks via Collaborative Multi-task Training
• Selective Question Answering under Domain Shift
• Noisy Computations during Inference: Harmful or Helpful?
• Evading Adversarial Example Detection Defenses with Orthogonal Projected Gradient Descent
• Towards Characterizing Adversarial Defects of Deep Learning Software from the Lens of Uncertainty
• $n$-ML: Mitigating Adversarial Examples via Ensembles of Topologically Manipulated Classifiers
• Adversarial Reinforcement Learning under Partial Observability in Autonomous Computer Network Defence
• Cassandra: Detecting Trojaned Networks from Adversarial Perturbations
• Random Directional Attack for Fooling Deep Neural Networks
• Stateful Detection of Black-Box Adversarial Attacks
• Defending against adversarial attacks on medical imaging AI system, classification or detection?
• TREATED:Towards Universal Defense against Textual Adversarial Attacks
• Regularizers for Single-step Adversarial Training
• Connecting the Dots: Detecting Adversarial Perturbations Using Context Inconsistency
• Subset Scanning Over Neural Network Activations
• BUZz: BUffer Zones for defending adversarial examples in image classification
• One Bit Matters: Understanding Adversarial Examples as the Abuse of Redundancy
• The Nonlinearity Coefficient - A Practical Guide to Neural Architecture Design
• Toward Metrics for Differentiating Out-of-Distribution Sets
• Most ReLU Networks Suffer from ${\ell }^2$ Adversarial Perturbations
• Drawing Robust Scratch Tickets: Subnetworks with Inborn Robustness Are Found within Randomly Initialized Networks
• Medical Aegis: Robust adversarial protectors for medical images
• Analytical Moment Regularizer for Gaussian Robust Networks
• Fine-grained Uncertainty Modeling in Neural Networks
• Feature-Filter: Detecting Adversarial Examples through Filtering off Recessive Features
• Technologies for Trustworthy Machine Learning: A Survey in a Socio-Technical Context
• Certifying Joint Adversarial Robustness for Model Ensembles
• OFEI: A Semi-black-box Android Adversarial Sample Attack Framework Against DLaaS
• Bandlimiting Neural Networks Against Adversarial Attacks
• Dompteur: Taming Audio Adversarial Examples
• Adversarial Examples in Multi-Layer Random ReLU Networks
• Unsupervised Detection of Adversarial Examples with Model Explanations
• Self-supervised Fine-tuning for Correcting Super-Resolution Convolutional Neural Networks
• Detecting Adversarial Patches with Class Conditional Reconstruction Networks
• Diminishing the Effect of Adversarial Perturbations via Refining Feature Representation
• Attack as Defense: Characterizing Adversarial Examples using Robustness
• Ensemble-in-One: Learning Ensemble within Random Gated Networks for Enhanced Adversarial Robustness
• Efficient detection of adversarial images
• A New Angle on L2 Regularization
• Moving Target Defense for Deep Visual Sensing against Adversarial Examples
• Clipping free attacks against artificial neural networks
• Deep Latent Defence
• Beating Attackers At Their Own Games: Adversarial Example Detection Using Adversarial Gradient Directions
• Models of Computational Profiles to Study the Likelihood of DNN Metamorphic Test Cases
• Robust Single-step Adversarial Training with Regularizer
• Adversarial Examples Detection beyond Image Space
• Improving White-box Robustness of Pre-processing Defenses via Joint Adversarial Training
• Using Anomaly Feature Vectors for Detecting, Classifying and Warning of Outlier Adversarial Examples
• Improving Calibration and Out-of-Distribution Detection in Medical Image Segmentation with Convolutional Neural Networks
• Immuno-mimetic Deep Neural Networks (Immuno-Net)
• A Survey on Assessing the Generalization Envelope of Deep Neural Networks: Predictive Uncertainty, Out-of-distribution and Adversarial Samples
• DetectX -- Adversarial Input Detection using Current Signatures in Memristive XBar Arrays
• A Hierarchical Feature Constraint to Camouflage Medical Adversarial Attacks
• Stochastic-Shield: A Probabilistic Approach Towards Training-Free Adversarial Defense in Quantized CNNs
• SPADE: A Spectral Method for Black-Box Adversarial Robustness Evaluation
• Mitigation of Adversarial Attacks through Embedded Feature Selection
• Interpretable Disentanglement of Neural Networks by Extracting Class-Specific Subnetwork
• A Useful Taxonomy for Adversarial Robustness of Neural Networks
• Detecting Adversarial Samples Using Density Ratio Estimates
• MixDefense: A Defense-in-Depth Framework for Adversarial Example Detection Based on Statistical and Semantic Analysis
• Out-distribution training confers robustness to deep neural networks
• On Lyapunov exponents and adversarial perturbation
• Analysis of Random Perturbations for Robust Convolutional Neural Networks
• Identification of Attack-Specific Signatures in Adversarial Examples
• Robustness from Simple Classifiers
• Exploiting Multi-Object Relationships for Detecting Adversarial Attacks in Complex Scenes
• Where Classification Fails, Interpretation Rises
• Adversarial Attacks on Deep Models for Financial Transaction Records
• Deep Bayesian Image Set Classification: A Defence Approach against Adversarial Attacks
• Compressive Sensing Based Adaptive Defence Against Adversarial Images
• Interpretable BoW Networks for Adversarial Example Detection
• Deterministic Certification to Adversarial Attacks via Bernstein Polynomial Approximation
• MadNet: Using a MAD Optimization for Defending Against Adversarial Attacks
• Learning to Separate Clusters of Adversarial Representations for Robust Adversarial Detection
• A Primer on Multi-Neuron Relaxation-based Adversarial Robustness Certification
• Interpreting Attributions and Interactions of Adversarial Attacks
• Likelihood Landscapes: A Unifying Principle Behind Many Adversarial Defenses
• Deep Adversarially-Enhanced k-Nearest Neighbors
• Learning to Disentangle Robust and Vulnerable Features for Adversarial Detection
• The Effect of Prior Lipschitz Continuity on the Adversarial Robustness of Bayesian Neural Networks
• Knowing More About Questions Can Help: Improving Calibration in Question Answering
• Using an ensemble color space model to tackle adversarial examples
• Defenses Against Multi-Sticker Physical Domain Attacks on Classifiers
• Accumulative Poisoning Attacks on Real-time Data
• Attack-agnostic Adversarial Detection on Medical Data Using Explainable Machine Learning
• BOSH: An Efficient Meta Algorithm for Decision-based Attacks
• Improving Compositionality of Neural Networks by Decoding Representations to Inputs
• Attribution of Gradient Based Adversarial Attacks for Reverse Engineering of Deceptions
• Detecting Adversaries, yet Faltering to Noise? Leveraging Conditional Variational AutoEncoders for Adversary Detection in the Presence of Noisy Images
• Adversarial Imitation Attack
• An Explainable Adversarial Robustness Metric for Deep Learning Neural Networks
• Weighted Average Precision: Adversarial Example Detection in the Visual Perception of Autonomous Vehicles
• SINVAD: Search-based Image Space Navigation for DNN Image Classifier Test Input Generation
• Leveraging Model Interpretability and Stability to increase Model Robustness
• Learning To Characterize Adversarial Subspaces
• Ensemble of Models Trained by Key-based Transformed Images for Adversarially Robust Defense Against Black-box Attacks