Generative Max-Mahalanobis Classifiers for Image Classification, Generation and More

Abstract

Joint Energy-based Model (JEM) of Grathwohl et al. shows that a standard softmax classifier can be reinterpreted as an energy-based model (EBM) for the joint distribution p(x,y); the resulting model can be optimized to improve calibration, robustness, and out-of-distribution detection, while generating samples rivaling the quality of recent GAN-based approaches. However, the softmax classifier that JEM exploits is inherently discriminative and its latent feature space is not well formulated as probabilistic distributions, which may hinder its potential for image generation and incur training instability. We hypothesize that generative classifiers, such as Linear Discriminant Analysis (LDA), might be more suitable for image generation since generative classifiers model the data generation process explicitly. This paper therefore investigates an LDA classifier for image classification and generation. In particular, the Max-Mahalanobis Classifier (MMC), a special case of LDA, fits our goal very well. We show that our Generative MMC (GMMC) can be trained discriminatively, generatively, or jointly for image classification and generation. Extensive experiments on multiple datasets show that GMMC achieves state-of-the-art discriminative and generative performances, while outperforming JEM in calibration, adversarial robustness, and out-of-distribution detection by a significant margin. Our source code is available at https://github.com/sndnyang/GMMC.

arXiv commentAccepted as a conference paper at ECML2021

References in corpus (15)

• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• On Calibration of Modern Neural Networks
• A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
• Robustness May Be at Odds with Accuracy
• A Fast and Simple Algorithm for Training Neural Probabilistic Language Models
• A Theory of Generative ConvNet
• Invertible Residual Networks
• Implicit Generation and Generalization in Energy-Based Models
• Rethinking Softmax Cross-Entropy Loss for Adversarial Robustness
• Image Synthesis with a Single (Robust) Classifier
• Do Deep Generative Models Know What They Don't Know?
• Detecting Out-of-Distribution Inputs to Deep Generative Models Using Typicality
• Max-Mahalanobis Linear Discriminant Analysis Networks
• Learning the Stein Discrepancy for Training and Evaluating Energy-Based Models without Sampling