Characterizing the Decision Boundary of Deep Neural Networks

Abstract

Deep neural networks and in particular, deep neural classifiers have become an integral part of many modern applications. Despite their practical success, we still have limited knowledge of how they work and the demand for such an understanding is evergrowing. In this regard, one crucial aspect of deep neural network classifiers that can help us deepen our knowledge about their decision-making behavior is to investigate their decision boundaries. Nevertheless, this is contingent upon having access to samples populating the areas near the decision boundary. To achieve this, we propose a novel approach we call Deep Decision boundary Instance Generation (DeepDIG). DeepDIG utilizes a method based on adversarial example generation as an effective way of generating samples near the decision boundary of any deep neural network model. Then, we introduce a set of important principled characteristics that take advantage of the generated instances near the decision boundary to provide multifaceted understandings of deep neural networks. We have performed extensive experiments on multiple representative datasets across various deep neural network models and characterized their decision boundaries. The code is publicly available at https://github.com/hamidkarimi/DeepDIG/.

Please contact the first author for any issue or the question regarding this paper

References in corpus (15)

• Adam: A Method for Stochastic Optimization
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• A Tutorial on Principal Component Analysis
• DeepXplore: Automated Whitebox Testing of Deep Learning Systems
• A guide to convolution arithmetic for deep learning
• Understanding Deep Convolutional Networks
• An Empirical Evaluation of Deep Learning on Highway Driving
• Adversarial Attacks on Deep Learning Models in Natural Language Processing: A Survey
• Adversarial Examples: Attacks and Defenses for Deep Learning
• Learning Hierarchical Discourse-level Structure for Fake News Detection
• On the Decision Boundary of Deep Neural Networks
• High Dimensional Spaces, Deep Learning and Adversarial Examples
• Deep Adversarial Network Alignment
• Investigating Decision Boundaries of Trained Neural Networks

Cited by in corpus (12)

• Analysis of Generalizability of Deep Neural Networks Based on the Complexity of Decision Boundary
• Understanding Deep Learning via Decision Boundary
• Qimera: Data-free Quantization with Synthetic Boundary Supporting Samples
• ANCER: Anisotropic Certification via Sample-wise Volume Maximization
• Concurrent Adversarial Learning for Large-Batch Training
• On the Difficulty of Membership Inference Attacks
• Human-in-the-loop model explanation via verbatim boundary identification in generated neighborhoods
• Extracting Global Dynamics of Loss Landscape in Deep Learning Models
• The Missing Margin: How Sample Corruption Affects Distance to the Boundary in ANNs
• Heating up decision boundaries: isocapacitory saturation, adversarial scenarios and generalization bounds
• Certainty Volume Prediction for Unsupervised Domain Adaptation
• How to Explain Neural Networks: an Approximation Perspective