Towards Proving the Adversarial Robustness of Deep Neural Networks

Abstract

Autonomous vehicles are highly complex systems, required to function reliably in a wide variety of situations. Manually crafting software controllers for these vehicles is difficult, but there has been some success in using deep neural networks generated using machine-learning. However, deep neural networks are opaque to human engineers, rendering their correctness very difficult to prove manually; and existing automated techniques, which were not designed to operate on neural networks, fail to scale to large systems. This paper focuses on proving the adversarial robustness of deep neural networks, i.e. proving that small perturbations to a correctly-classified input to the network cannot cause it to be misclassified. We describe some of our recent and ongoing work on verifying the adversarial robustness of networks, and discuss some of the open questions we have encountered and how they might be addressed.

arXiv commentIn Proceedings FVAV 2017, arXiv:1709.02126

References in corpus (2)

• Intriguing properties of neural networks
• End to End Learning for Self-Driving Cars

Cited by in corpus (40)

• Certified Defenses against Adversarial Examples
• Certifying Some Distributional Robustness with Principled Adversarial Training
• Securing Connected & Autonomous Vehicles: Challenges Posed by Adversarial Machine Learning and The Way Forward
• Adversarial Examples: Attacks and Defenses for Deep Learning
• Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey
• Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach
• Adversarial Sample Detection for Deep Neural Network through Model Mutation Testing
• AI Safety Gridworlds
• Benchmarking Neural Network Robustness to Common Corruptions and Surface Variations
• Provably Minimally-Distorted Adversarial Examples
• An Abstraction-Based Framework for Neural Network Verification
• Analyzing the Robustness of Nearest Neighbors to Adversarial Examples
• DeepSafe: A Data-driven Approach for Checking Adversarial Robustness in Neural Networks
• Combinatorial Testing for Deep Learning Systems
• Testing and verification of neural-network-based safety-critical control software: A systematic literature review
• Detecting Adversarial Samples for Deep Neural Networks through Mutation Testing
• Parallelization Techniques for Verifying Neural Networks
• Formal Verification of Decision-Tree Ensemble Model and Detection of its Violating-input-value Ranges
• Gradient Band-based Adversarial Training for Generalized Attack Immunity of A3C Path Finding
• Towards Security Threats of Deep Learning Systems: A Survey
• Towards Scalable Verification of Deep Reinforcement Learning
• Interpreting Neural Networks Using Flip Points
• Engineering problems in machine learning systems
• Revisiting Deep Neural Network Test Coverage from the Test Effectiveness Perspective
• Safe Predictors for Enforcing Input-Output Specifications
• RobOT: Robustness-Oriented Testing for Deep Learning Systems
• Pruning and Slicing Neural Networks using Formal Verification
• Formal methods and software engineering for DL. Security, safety and productivity for DL systems development
• Adversarial Attack Type I: Cheat Classifiers by Significant Changes
• ROBY: Evaluating the Robustness of a Deep Model by its Decision Boundaries
• Sample Complexity of Adversarially Robust Linear Classification on Separated Data
• When are Non-Parametric Methods Robust?
• Simplifying Neural Networks using Formal Verification
• Consistent Non-Parametric Methods for Maximizing Robustness
• Uncovering Discrimination Clusters: Quantifying and Explaining Systematic Fairness Violations
• Deterministic Certification to Adversarial Attacks via Bernstein Polynomial Approximation
• Boosting the Robustness Verification of DNN by Identifying the Achilles's Heel
• Improving Adversarial Robustness by Encouraging Discriminative Features
• Exploiting Vulnerability of Pooling in Convolutional Neural Networks by Strict Layer-Output Manipulation for Adversarial Attacks
• Using Quantifier Elimination to Enhance the Safety Assurance of Deep Neural Networks