Probing the Purview of Neural Networks via Gradient Analysis
arXiv:2304.02834 · doi:10.1109/ACCESS.2023.3263210
Abstract
We analyze the data-dependent capacity of neural networks and assess anomalies in inputs from the perspective of networks during inference. The notion of data-dependent capacity allows for analyzing the knowledge base of a model populated by learned features from training data. We define purview as the additional capacity necessary to characterize inference samples that differ from the training data. To probe the purview of a network, we utilize gradients to measure the amount of change required for the model to characterize the given inputs more accurately. To eliminate the dependency on ground-truth labels in generating gradients, we introduce confounding labels that are formulated by combining multiple categorical labels. We demonstrate that our gradient-based approach can effectively differentiate inputs that cannot be accurately represented with learned features. We utilize our approach in applications of detecting anomalous inputs, including out-of-distribution, adversarial, and corrupted samples. Our approach requires no hyperparameter tuning or additional data processing and outperforms state-of-the-art methods by up to 2.7%, 19.8%, and 35.6% of AUROC scores, respectively.
arXiv commentPublished in IEEE Access. 17 pages, 6 figures
References in corpus (21)
- Explaining and Harnessing Adversarial Examples
- mixup: Beyond Empirical Risk Minimization
- Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks
- On Calibration of Modern Neural Networks
- A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
- LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
- A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
- Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
- Adversarially Robust Generalization Requires More Data
- Energy-based Out-of-distribution Detection
- Learning Confidence for Out-of-Distribution Detection in Neural Networks
- CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances
- Intrinsic dimension of data representations in deep neural networks
- In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learning
- A New Defense Against Adversarial Images: Turning a Weakness into a Strength
- Big Neural Networks Waste Capacity
- UNIQUE: Unsupervised Image Quality Estimation
- CURE-TSR: Challenging Unreal and Real Environments for Traffic Sign Recognition
- Learning a Deep ConvNet for Multi-label Classification with Partial Labels
- Introspective Learning : A Two-Stage Approach for Inference in Neural Networks
- Gradient-Based Adversarial and Out-of-Distribution Detection