Selective Classification for Deep Neural Networks

Abstract

Selective classification techniques (also known as reject option) have not yet been considered in the context of deep neural networks (DNNs). These techniques can potentially significantly improve DNNs prediction performance by trading-off coverage. In this paper we propose a method to construct a selective classifier given a trained neural network. Our method allows a user to set a desired risk level. At test time, the classifier rejects instances as needed, to grant the desired risk (with high probability). Empirical results over CIFAR and ImageNet convincingly demonstrate the viability of our method, which opens up possibilities to operate DNNs in mission-critical applications. For example, using our method an unprecedented 2% error in top-5 ImageNet classification can be guaranteed with probability 99.9%, and almost 60% test coverage.

References in corpus (1)

• Wide Residual Networks

Cited by in corpus (18)

• Recent Advances in Open Set Recognition: A Survey
• Curriculum Learning for Speech Emotion Recognition from Crowdsourced Labels
• Run-Time Monitoring of Machine Learning for Robotic Perception: A Survey of Emerging Trends
• Uncertainty Aware Training to Improve Deep Learning Model Calibration for Classification of Cardiac MR Images
• Dynamically Sacrificing Accuracy for Reduced Computation: Cascaded Inference Based on Softmax Confidence
• Into the Unknown: Active Monitoring of Neural Networks
• Outside the Box: Abstraction-Based Monitoring of Neural Networks
• A Unified Benchmark for the Unknown Detection Capability of Deep Neural Networks
• Controlling Risk of Web Question Answering
• Better the Devil you Know: An Analysis of Evasion Attacks using Out-of-Distribution Adversarial Examples
• Utilizing Network Properties to Detect Erroneous Inputs
• Meta-Cal: Well-controlled Post-hoc Calibration by Ranking
• Generator evaluator-selector net for panoptic image segmentation and splitting unfamiliar objects into parts
• SIPA: A Simple Framework for Efficient Networks
• Evolutionary Optimization of High-Coverage Budgeted Classifiers
• Improving Compositionality of Neural Networks by Decoding Representations to Inputs
• Selective Probabilistic Classifier Based on Hypothesis Testing
• When In Doubt, Abstain: The Impact of Abstention on Strategic Classification