Reject Illegal Inputs with Generative Classifier Derived from Any Discriminative Classifier

Abstract

Generative classifiers have been shown promising to detect illegal inputs including adversarial examples and out-of-distribution samples. Supervised Deep Infomax~(SDIM) is a scalable end-to-end framework to learn generative classifiers. In this paper, we propose a modification of SDIM termed SDIM-\emph{logit}. Instead of training generative classifier from scratch, SDIM-\emph{logit} first takes as input the logits produced any given discriminative classifier, and generate logit representations; then a generative classifier is derived by imposing statistical constraints on logit representations. SDIM-\emph{logit} could inherit the performance of the discriminative classifier without loss. SDIM-\emph{logit} incurs a negligible number of additional parameters, and can be efficiently trained with base classifiers fixed. We perform \emph{classification with rejection}, where test samples whose class conditionals are smaller than pre-chosen thresholds will be rejected without predictions. Experiments on illegal inputs, including adversarial examples, samples with common corruptions, and out-of-distribution~(OOD) samples show that allowed to reject a portion of test samples, SDIM-\emph{logit} significantly improves the performance on the left test sets.

arXiv comment7 pages, 3 figures

References in corpus (9)

• Towards Deep Learning Models Resistant to Adversarial Attacks
• Fast is better than free: Revisiting adversarial training
• PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examples
• Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Models
• Adversarial Robustness through Local Linearization
• WAIC, but Why? Generative Ensembles for Robust Anomaly Detection
• Adversarial Examples Are a Natural Consequence of Test Error in Noise
• Adversarial Examples Improve Image Recognition
• Are Generative Classifiers More Robust to Adversarial Attacks?