Robust Physical-World Attacks on Deep Learning Models

Abstract

Recent studies show that the state-of-the-art deep neural networks (DNNs) are vulnerable to adversarial examples, resulting from small-magnitude perturbations added to the input. Given that that emerging physical systems are using DNNs in safety-critical situations, adversarial examples could mislead these systems and cause dangerous situations.Therefore, understanding adversarial examples in the physical world is an important step towards developing resilient learning algorithms. We propose a general attack algorithm,Robust Physical Perturbations (RP2), to generate robust visual adversarial perturbations under different physical conditions. Using the real-world case of road sign classification, we show that adversarial examples generated using RP2 achieve high targeted misclassification rates against standard-architecture road sign classifiers in the physical world under various environmental conditions, including viewpoints. Due to the current lack of a standardized testing method, we propose a two-stage evaluation methodology for robust physical adversarial examples consisting of lab and field tests. Using this methodology, we evaluate the efficacy of physical adversarial manipulations on real objects. Witha perturbation in the form of only black and white stickers,we attack a real stop sign, causing targeted misclassification in 100% of the images obtained in lab settings, and in 84.8%of the captured video frames obtained on a moving vehicle(field test) for the target classifier.

Accepted to CVPR 2018

References in corpus (10)

• Delving into Transferable Adversarial Examples and Black-box Attacks
• Stealing Machine Learning Models via Prediction APIs
• Synthesizing Robust Adversarial Examples
• NO Need to Worry about Adversarial Examples in Object Detection in Autonomous Vehicles
• Houdini: Fooling Deep Structured Prediction Models
• Delving into adversarial attacks on deep policies
• Adversarial Examples for Semantic Segmentation and Object Detection
• Universal adversarial perturbations
• Note on Attacking Object Detectors with Adversarial Stickers
• Adversarial examples for generative models

Cited by in corpus (180)

• ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models
• The Zwicky Transient Facility: Science Objectives
• Deep neural network models for computational histopathology: A survey
• Towards Explainable Artificial Intelligence
• Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality
• Spectral Signatures in Backdoor Attacks
• Robustness May Be at Odds with Accuracy
• AdvHat: Real-world adversarial attack on ArcFace Face ID system
• Generating Adversarial Examples with Adversarial Networks
• Adversarial Examples: Attacks and Defenses for Deep Learning
• DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Cars
• Galaxy Zoo DECaLS: Detailed Visual Morphology Measurements from Volunteers and Deep Learning for 314,000 Galaxies
• Why do deep convolutional networks generalize so poorly to small image transformations?
• Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey
• Adversarial Machine Learning in Image Classification: A Survey Towards the Defender's Perspective
• Understanding Membership Inferences on Well-Generalized Learning Models
• Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach
• TABOR: A Highly Accurate Approach to Inspecting and Restoring Trojan Backdoors in AI Systems
• The Robust Manifold Defense: Adversarial Training using Generative Models
• Adversarial Attacks Against Medical Deep Learning Systems
• Benchmarking Neural Network Robustness to Common Corruptions and Surface Variations
• Adversarial Examples that Fool Detectors
• Adv-BNN: Improved Adversarial Defense through Robust Bayesian Neural Network
• Optimization and Abstraction: A Synergistic Approach for Analyzing Neural Network Robustness
• POBA-GA: Perturbation Optimized Black-Box Adversarial Attacks via Genetic Algorithm
• Testing Robustness Against Unforeseen Adversaries
• Robust Adversarial Perturbation on Deep Proposal-based Models
• CommanderSong: A Systematic Approach for Practical Adversarial Voice Recognition
• Invisible Mask: Practical Attacks on Face Recognition with Infrared
• Adversarial Objects Against LiDAR-Based Autonomous Driving Systems
• EAD: Elastic-Net Attacks to Deep Neural Networks via Adversarial Examples
• Software Engineering for AI-Based Systems: A Survey
• Combinatorial Testing for Deep Learning Systems
• Convergence of Adversarial Training in Overparametrized Neural Networks
• Standard detectors aren't (currently) fooled by physical adversarial stop signs
• PoTrojan: powerful neural-level trojan designs in deep learning models
• Shield: Fast, Practical Defense and Vaccination for Deep Learning using JPEG Compression
• Experimental Resilience Assessment of An Open-Source Driving Agent
• Fault Sneaking Attack: a Stealthy Framework for Misleading Deep Neural Networks
• Adversarial Attacks on Time-Series Intrusion Detection for Industrial Control Systems
• Adversarial Examples - A Complete Characterisation of the Phenomenon
• Defending Against Universal Attacks Through Selective Feature Regeneration
• Adversarially Robust Few-Shot Learning: A Meta-Learning Approach
• Indirect Adversarial Attacks via Poisoning Neighbors for Graph Convolutional Networks
• Physical Adversarial Attack on Vehicle Detector in the Carla Simulator
• Adversarial T-shirt! Evading Person Detectors in A Physical World
• Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networks
• Robustness Verification of Tree-based Models
• Note on Attacking Object Detectors with Adversarial Stickers
• Enhancing Adversarial Example Transferability with an Intermediate Level Attack
• Fooling OCR Systems with Adversarial Text Images
• Daedalus: Breaking Non-Maximum Suppression in Object Detection via Adversarial Examples
• Label Smoothing and Logit Squeezing: A Replacement for Adversarial Training?
• Defense against Adversarial Attacks in NLP via Dirichlet Neighborhood Ensemble
• Piracy Resistant Watermarks for Deep Neural Networks
• Transferable Adversarial Attacks for Image and Video Object Detection
• Universal Adversarial Perturbations: A Survey
• StegaStamp: Invisible Hyperlinks in Physical Photographs
• SLAP: Improving Physical Adversarial Examples with Short-Lived Adversarial Perturbations
• Towards a Robust Deep Neural Network in Texts: A Survey
• Adversarial Examples in Deep Learning: Characterization and Divergence
• Adversarial Examples in RF Deep Learning: Detection of the Attack and its Physical Robustness
• Exploring Connections Between Active Learning and Model Extraction
• Adversarial Reprogramming of Neural Networks
• MobilBye: Attacking ADAS with Camera Spoofing
• Generating 3D Adversarial Point Clouds
• Towards Imperceptible and Robust Adversarial Example Attacks against Neural Networks
• Fooling Detection Alone is Not Enough: First Adversarial Attack against Multiple Object Tracking
• Adversarial Laser Beam: Effective Physical-World Attack to DNNs in a Blink
• Adversarial Camouflage: Hiding Physical-World Attacks with Natural Styles
• On the Adversarial Robustness of Subspace Learning
• Adversarial Attacks, Regression, and Numerical Stability Regularization
• Adversarial attacks on Copyright Detection Systems
• Clean-Label Backdoor Attacks on Video Recognition Models
• Simple Physical Adversarial Examples against End-to-End Autonomous Driving Models
• Poison as a Cure: Detecting & Neutralizing Variable-Sized Backdoor Attacks in Deep Neural Networks
• Scratch that! An Evolution-based Adversarial Attack against Neural Networks
• Seeing isn't Believing: Practical Adversarial Attack Against Object Detectors
• Exploring Adversarial Attack in Spiking Neural Networks with Spike-Compatible Gradient
• A Provable Defense for Deep Residual Networks
• Improving Robustness of Deep-Learning-Based Image Reconstruction
• A Spectral View of Adversarially Robust Features
• On the Design of Black-box Adversarial Examples by Leveraging Gradient-free Optimization and Operator Splitting Method
• Making an Invisibility Cloak: Real World Adversarial Attacks on Object Detectors
• Robustifying Models Against Adversarial Attacks by Langevin Dynamics
• A Little Fog for a Large Turn
• VerIDeep: Verifying Integrity of Deep Neural Networks through Sensitive-Sample Fingerprinting
• Bias-based Universal Adversarial Patch Attack for Automatic Check-out
• Adversarial Image Color Transformations in Explicit Color Filter Space
• Adversarial Resilience Learning - Towards Systemic Vulnerability Analysis for Large and Complex Systems
• Noise Sensitivity-Based Energy Efficient and Robust Adversary Detection in Neural Networks
• On the Limitation of MagNet Defense against $L_1$-based Adversarial Examples
• Adversarial Learning of Deepfakes in Accounting
• Adversarial Examples on Object Recognition: A Comprehensive Survey
• Projecting Trouble: Light Based Adversarial Attacks on Deep Learning Classifiers
• AdaCompress: Adaptive Compression for Online Computer Vision Services
• Defense against adversarial attacks on deep convolutional neural networks through nonlocal denoising
• Adversarial Attacks Beyond the Image Space
• Efficient Adversarial Training with Transferable Adversarial Examples
• Towards Large yet Imperceptible Adversarial Image Perturbations with Perceptual Color Distance
• Incorporating Unlabeled Data into Distributionally Robust Learning
• Towards Robust Deep Neural Networks
• Attacking Vision-based Perception in End-to-End Autonomous Driving Models
• Building Robust Deep Neural Networks for Road Sign Detection
• Geometric robustness of deep networks: analysis and improvement
• How intelligent are convolutional neural networks?
• Query-Efficient Black-Box Attack by Active Learning
• FaceLeaks: Inference Attacks against Transfer Learning Models via Black-box Queries
• Playing it Safe: Adversarial Robustness with an Abstain Option
• Category-wise Attack: Transferable Adversarial Examples for Anchor Free Object Detection
• Hu-Fu: Hardware and Software Collaborative Attack Framework against Neural Networks
• Monocular Depth Estimators: Vulnerabilities and Attacks
• Exploring Adversarial Examples: Patterns of One-Pixel Attacks
• AI Enabling Technologies: A Survey
• Do We Really Need to Learn Representations from In-domain Data for Outlier Detection?
• Robust Semantic Segmentation with Superpixel-Mix
• MeshAdv: Adversarial Meshes for Visual Recognition
• Rearchitecting Classification Frameworks For Increased Robustness
• Harmonic Adversarial Attack Method
• IoU Attack: Towards Temporally Coherent Black-Box Adversarial Attack for Visual Object Tracking
• GRIP: Generative Robust Inference and Perception for Semantic Robot Manipulation in Adversarial Environments
• Detection as Regression: Certified Object Detection by Median Smoothing
• QEBA: Query-Efficient Boundary-Based Blackbox Attack
• Structure-Preserving Transformation: Generating Diverse and Transferable Adversarial Examples
• Identifying Model Weakness with Adversarial Examiner
• T3: Tree-Autoencoder Constrained Adversarial Text Generation for Targeted Attack
• One Bit Matters: Understanding Adversarial Examples as the Abuse of Redundancy
• PhysGAN: Generating Physical-World-Resilient Adversarial Examples for Autonomous Driving
• Intermediate Level Adversarial Attack for Enhanced Transferability
• Stealthy and Efficient Adversarial Attacks against Deep Reinforcement Learning
• Adversarial Robustness Study of Convolutional Neural Network for Lumbar Disk Shape Reconstruction from MR images
• Certifying Joint Adversarial Robustness for Model Ensembles
• Evolutionary Computation and AI Safety: Research Problems Impeding Routine and Safe Real-world Application of Evolution
• LanCe: A Comprehensive and Lightweight CNN Defense Methodology against Physical Adversarial Attacks on Embedded Multimedia Applications
• Vision-based Navigation of Autonomous Vehicle in Roadway Environments with Unexpected Hazards
• ASP:A Fast Adversarial Attack Example Generation Framework based on Adversarial Saliency Prediction
• Towards an Understanding of Neural Networks in Natural-Image Spaces
• Robust Tracking against Adversarial Attacks
• Techniques for Adversarial Examples Threatening the Safety of Artificial Intelligence Based Systems
• Structure-Invariant Testing for Machine Translation
• The Helmholtz Method: Using Perceptual Compression to Reduce Machine Learning Complexity
• Hacking Neural Networks: A Short Introduction
• DoPa: A Comprehensive CNN Detection Methodology against Physical Adversarial Attacks
• Efficient detection of adversarial images
• A New Angle on L2 Regularization
• Automated Testing for Deep Learning Systems with Differential Behavior Criteria
• Towards Adversarial Configurations for Software Product Lines
• Adversarial Robustness Guarantees for Random Deep Neural Networks
• On the Similarity of Deep Learning Representations Across Didactic and Adversarial Examples
• Visually Imperceptible Adversarial Patch Attacks on Digital Images
• Moving Target Defense for Deep Visual Sensing against Adversarial Examples
• ExAD: An Ensemble Approach for Explanation-based Adversarial Detection
• Fixed Points in Cyber Space: Rethinking Optimal Evasion Attacks in the Age of AI-NIDS
• FineFool: Fine Object Contour Attack via Attention
• A principled approach for generating adversarial images under non-smooth dissimilarity metrics
• Search Space of Adversarial Perturbations against Image Filters
• Augmenting Model Robustness with Transformation-Invariant Attacks
• Verisimilar Percept Sequences Tests for Autonomous Driving Intelligent Agent Assessment
• Black-box Adversarial Sample Generation Based on Differential Evolution
• The Human Visual System and Adversarial AI
• Non-Determinism in Neural Networks for Adversarial Robustness
• Unifying Bilateral Filtering and Adversarial Training for Robust Neural Networks
• Using Randomness to Improve Robustness of Machine-Learning Models Against Evasion Attacks
• Adversarial Examples in Remote Sensing
• Local Lipschitz Constant Computation of ReLU-FNNs: Upper Bound Computation with Exactness Verification
• Causality and Generalizability: Identifiability and Learning Methods
• Fast Local Attack: Generating Local Adversarial Examples for Object Detectors
• An Empirical Review of Adversarial Defenses
• Universal Physical Camouflage Attacks on Object Detectors
• Towards Quality Assurance of Software Product Lines with Adversarial Configurations
• Moiré Attack (MA): A New Potential Risk of Screen Photos
• Robust Assessment of Real-World Adversarial Examples
• Adversarial Data Encryption
• Adversarial Ranking Attack and Defense
• Adversarial Examples and the Deeper Riddle of Induction: The Need for a Theory of Artifacts in Deep Learning
• Testing Machine Translation via Referential Transparency
• Open-set Adversarial Defense
• Are You Tampering With My Data?
• Multimedia Edge Computing
• Law and Adversarial Machine Learning