Safer Classification by Synthesis

Abstract

The discriminative approach to classification using deep neural networks has become the de-facto standard in various fields. Complementing recent reservations about safety against adversarial examples, we show that conventional discriminative methods can easily be fooled to provide incorrect labels with very high confidence to out of distribution examples. We posit that a generative approach is the natural remedy for this problem, and propose a method for classification using generative models. At training time, we learn a generative model for each class, while at test time, given an example to classify, we query each generator for its most similar generation, and select the class corresponding to the most similar one. Our approach is general and can be used with expressive models such as GANs and VAEs. At test time, our method accurately "knows when it does not know," and provides resilience to out of distribution examples while maintaining competitive performance for standard examples.

References in corpus (5)

• Progressive Growing of GANs for Improved Quality, Stability, and Variation
• BEGAN: Boundary Equilibrium Generative Adversarial Networks
• Towards Principled Methods for Training Generative Adversarial Networks
• Distance-based Confidence Score for Neural Network Classifiers
• Novelty Detection with GAN

Cited by in corpus (13)

• How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review
• Out-of-distribution Detection in Classifiers via Generation
• Detecting Out-of-Distribution Inputs in Deep Neural Networks Using an Early-Layer Output
• A Less Biased Evaluation of Out-of-distribution Sample Detectors
• Confidence from Invariance to Image Transformations
• Analysis of Confident-Classifiers for Out-of-distribution Detection
• Towards neural networks that provably know when they don't know
• Deep Variational Semi-Supervised Novelty Detection
• Bayesian OOD detection with aleatoric uncertainty and outlier exposure
• Online Safety Assurance for Deep Reinforcement Learning
• Deep Residual Flow for Out of Distribution Detection
• The Effect of Optimization Methods on the Robustness of Out-of-Distribution Detection Approaches
• FROB: Few-shot ROBust Model for Classification and Out-of-Distribution Detection