Investigating Decision Boundaries of Trained Neural Networks

Abstract

Deep learning models have been the subject of study from various perspectives, for example, their training process, interpretation, generalization error, robustness to adversarial attacks, etc. A trained model is defined by its decision boundaries, and therefore, many of the studies about deep learning models speculate about the decision boundaries, and sometimes make simplifying assumptions about them. So far, finding exact points on the decision boundaries of trained deep models has been considered an intractable problem. Here, we compute exact points on the decision boundaries of these models and provide mathematical tools to investigate the surfaces that define the decision boundaries. Through numerical results, we confirm that some of the speculations about the decision boundaries are accurate, some of the computational methods can be improved, and some of the simplifying assumptions may be unreliable, for models with nonlinear activation functions. We advocate for verification of simplifying assumptions and approximation methods, wherever they are used. Finally, we demonstrate that the computational practices used for finding adversarial examples can be improved and computing the closest point on the decision boundary reveals the weakest vulnerability of a model against adversarial attack.

References in corpus (6)

• Actionable Recourse in Linear Classification
• Adversarial Examples Are Not Bugs, They Are Features
• A Boundary Tilting Persepective on the Phenomenon of Adversarial Examples
• Predicting the Generalization Gap in Deep Networks with Margin Distributions
• A Simple Explanation for the Existence of Adversarial Examples with Small Hamming Distance
• Interpreting Neural Networks Using Flip Points

Cited by in corpus (6)

• Characterizing the Decision Boundary of Deep Neural Networks
• Analysis of Generalizability of Deep Neural Networks Based on the Complexity of Decision Boundary
• Auditing and Debugging Deep Learning Models via Decision Boundaries: Individual-level and Group-level Analysis
• ROBY: Evaluating the Robustness of a Deep Model by its Decision Boundaries
• Using Wavelets to Analyze Similarities in Image-Classification Datasets
• Fractal Dimension Generalization Measure