NO Need to Worry about Adversarial Examples in Object Detection in Autonomous Vehicles

Abstract

It has been shown that most machine learning algorithms are susceptible to adversarial perturbations. Slightly perturbing an image in a carefully chosen direction in the image space may cause a trained neural network model to misclassify it. Recently, it was shown that physical adversarial examples exist: printing perturbed images then taking pictures of them would still result in misclassification. This raises security and safety concerns. However, these experiments ignore a crucial property of physical objects: the camera can view objects from different distances and at different angles. In this paper, we show experiments that suggest that current constructions of physical adversarial examples do not disrupt object detection from a moving platform. Instead, a trained neural network classifies most of the pictures taken from different distances and angles of a perturbed image correctly. We believe this is because the adversarial property of the perturbation is sensitive to the scale at which the perturbed picture is viewed, so (for example) an autonomous car will misclassify a stop sign only from a small range of distances. Our work raises an important question: can one construct examples that are adversarial for many or most viewing conditions? If so, the construction should offer very significant insights into the internal representation of patterns by deep networks. If not, there is a good prospect that adversarial examples can be reduced to a curiosity with little practical impact.

Accepted to CVPR 2017, Spotlight Oral Workshop

References in corpus (3)

• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
• Towards Deep Neural Network Architectures Robust to Adversarial Examples
• Defensive Distillation is Not Robust to Adversarial Examples

Cited by in corpus (62)

• Synthesizing Robust Adversarial Examples
• Robust Physical-World Attacks on Deep Learning Models
• Fast is better than free: Revisiting adversarial training
• Countering Adversarial Images using Input Transformations
• Adversarial Examples: Opportunities and Challenges
• Regularization for Deep Learning: A Taxonomy
• Adversarial Example Detection for DNN Models: A Review and Experimental Comparison
• Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples
• Adversarial Attacks Against Medical Deep Learning Systems
• Adversarial Examples that Fool Detectors
• Physical Adversarial Examples for Object Detectors
• Understanding and Enhancing the Transferability of Adversarial Examples
• Traffic Sign Detection under Challenging Conditions: A Deeper Look Into Performance Variations and Spectral Characteristics
• F1/10: An Open-Source Autonomous Cyber-Physical Platform
• Adversarial Robustness Against the Union of Multiple Perturbation Models
• Adversarial camera stickers: A physical camera-based attack on deep learning systems
• A Survey of Safety and Trustworthiness of Deep Neural Networks: Verification, Testing, Adversarial Attack and Defence, and Interpretability
• Adversarial Examples in Modern Machine Learning: A Review
• Standard detectors aren't (currently) fooled by physical adversarial stop signs
• Overfitting in adversarially robust deep learning
• Adversarial Patch Attacks on Monocular Depth Estimation Networks
• Provably Robust Boosted Decision Stumps and Trees against Adversarial Attacks
• Verifying Controllers with Convolutional Neural Network-based Perception: A Case for Intelligible, Safe, and Precise Abstractions
• Safeguarding connected autonomous vehicle communication: Protocols, intra- and inter-vehicular attacks and defenses
• Dynamic Adversarial Patch for Evading Object Detection Models
• Traffic Signs in the Wild: Highlights from the IEEE Video and Image Processing Cup 2017 Student Competition [SP Competitions]
• Fooling a Real Car with Adversarial Traffic Signs
• Challenging Environments for Traffic Sign Detection: Reliability Assessment under Inclement Conditions
• ConAML: Constrained Adversarial Machine Learning for Cyber-Physical Systems
• DUP-Net: Denoiser and Upsampler Network for 3D Adversarial Point Clouds Defense
• Adversarial attacks on Copyright Detection Systems
• Seeing isn't Believing: Practical Adversarial Attack Against Object Detectors
• Simple Physical Adversarial Examples against End-to-End Autonomous Driving Models
• PatchCensor: Patch Robustness Certification for Transformers via Exhaustive Testing
• ATHENA: A Framework based on Diverse Weak Defenses for Building Adversarial Defense
• Retrieval-Augmented Convolutional Neural Networks for Improved Robustness against Adversarial Examples
• Attacking Vision-based Perception in End-to-End Autonomous Driving Models
• Building Robust Deep Neural Networks for Road Sign Detection
• APRICOT: A Dataset of Physical Adversarial Attacks on Object Detection
• Exact Adversarial Attack to Image Captioning via Structured Output Learning with Latent Variables
• Cooling-Shrinking Attack: Blinding the Tracker with Imperceptible Noises
• Built-in Vulnerabilities to Imperceptible Adversarial Perturbations
• On the Robustness of Human Pose Estimation
• DriveGuard: Robustification of Automated Driving Systems with Deep Spatio-Temporal Convolutional Autoencoder
• PhysGAN: Generating Physical-World-Resilient Adversarial Examples for Autonomous Driving
• Attacking CNN-based anti-spoofing face authentication in the physical domain
• Taking Care of The Discretization Problem: A Comprehensive Study of the Discretization Problem and A Black-Box Adversarial Attack in Discrete Integer Domain
• Corner Case Generation and Analysis for Safety Assessment of Autonomous Vehicles
• Formal methods and software engineering for DL. Security, safety and productivity for DL systems development
• Butterfly Effect: Bidirectional Control of Classification Performance by Small Additive Perturbation
• Robust Synthesis of Adversarial Visual Examples Using a Deep Image Prior
• CCA: Exploring the Possibility of Contextual Camouflage Attack on Object Detection
• Can 3D Adversarial Logos Cloak Humans?
• Who is Responsible for Adversarial Defense?
• The Effects of Image Distribution and Task on Adversarial Robustness
• Learning Transferable 3D Adversarial Cloaks for Deep Trained Detectors
• Generating Adversarial Inputs Using A Black-box Differential Technique
• Controlled Caption Generation for Images Through Adversarial Attacks
• Adversarial Robustness of Deep Sensor Fusion Models
• Attack-agnostic Adversarial Detection on Medical Data Using Explainable Machine Learning
• Messing Up 3D Virtual Environments: Transferable Adversarial 3D Objects
• Adversarial Ranking Attack and Defense