Probing the Purview of Neural Networks via Gradient Analysis

Abstract

We analyze the data-dependent capacity of neural networks and assess anomalies in inputs from the perspective of networks during inference. The notion of data-dependent capacity allows for analyzing the knowledge base of a model populated by learned features from training data. We define purview as the additional capacity necessary to characterize inference samples that differ from the training data. To probe the purview of a network, we utilize gradients to measure the amount of change required for the model to characterize the given inputs more accurately. To eliminate the dependency on ground-truth labels in generating gradients, we introduce confounding labels that are formulated by combining multiple categorical labels. We demonstrate that our gradient-based approach can effectively differentiate inputs that cannot be accurately represented with learned features. We utilize our approach in applications of detecting anomalous inputs, including out-of-distribution, adversarial, and corrupted samples. Our approach requires no hyperparameter tuning or additional data processing and outperforms state-of-the-art methods by up to 2.7%, 19.8%, and 35.6% of AUROC scores, respectively.

Published in IEEE Access. 17 pages, 6 figures

References in corpus (8)

• Explaining and Harnessing Adversarial Examples
• On Calibration of Modern Neural Networks
• In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learning
• A New Defense Against Adversarial Images: Turning a Weakness into a Strength
• Big Neural Networks Waste Capacity
• Learning a Deep ConvNet for Multi-label Classification with Partial Labels
• Introspective Learning : A Two-Stage Approach for Inference in Neural Networks
• Gradient-Based Adversarial and Out-of-Distribution Detection