A Statistical Approach to Assessing Neural Network Robustness

Abstract

We present a new approach to assessing the robustness of neural networks based on estimating the proportion of inputs for which a property is violated. Specifically, we estimate the probability of the event that the property is violated under an input model. Our approach critically varies from the formal verification framework in that when the property can be violated, it provides an informative notion of how robust the network is, rather than just the conventional assertion that the network is not verifiable. Furthermore, it provides an ability to scale to larger networks than formal verification approaches. Though the framework still provides a formal guarantee of satisfiability whenever it successfully finds one or more violations, these advantages do come at the cost of only providing a statistical estimate of unsatisfiability whenever no violation is found. Key to the practical success of our approach is an adaptation of multi-level splitting, a Monte Carlo approach for estimating the probability of rare events, to our statistical robustness framework. We demonstrate that our approach is able to emulate formal verification procedures on benchmark problems, while scaling to larger networks and providing reliable additional information in the form of accurate estimates of the violation probability.

arXiv commentTo appear at the 7th International Conference on Learning Representations (ICLR 2019), New Orleans

References in corpus (7)

• Densely Connected Convolutional Networks
• Towards Deep Learning Models Resistant to Adversarial Attacks
• Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach
• Towards Fast Computation of Certified Robustness for ReLU Networks
• Rigorous Agent Evaluation: An Adversarial Approach to Uncover Catastrophic Failures
• Gradient Masking Causes CLEVER to Overestimate Adversarial Perturbation Size
• Verification of Binarized Neural Networks via Inter-Neuron Factoring

Cited by in corpus (8)

• Neural Bridge Sampling for Evaluating Safety-Critical Autonomous Systems
• Embedding and Extraction of Knowledge in Tree Ensemble Classifiers
• Improving the Robustness of Transformer-based Large Language Models with Dynamic Attention
• Uncertainty Quantification and Confidence Intervals for Naive Rare-Event Estimators
• Assessing the Reliability of Deep Learning Classifiers Through Robustness Evaluation and Operational Profiles
• Certifying Neural Network Robustness to Random Input Noise from Samples
• Over-Conservativeness of Variance-Based Efficiency Criteria and Probabilistic Efficiency in Rare-Event Simulation
• Deep Probabilistic Accelerated Evaluation: A Robust Certifiable Rare-Event Simulation Methodology for Black-Box Safety-Critical Systems