Max-Mahalanobis Linear Discriminant Analysis Networks

Abstract

A deep neural network (DNN) consists of a nonlinear transformation from an input to a feature representation, followed by a common softmax linear classifier. Though many efforts have been devoted to designing a proper architecture for nonlinear transformation, little investigation has been done on the classifier part. In this paper, we show that a properly designed classifier can improve robustness to adversarial attacks and lead to better prediction results. Specifically, we define a Max-Mahalanobis distribution (MMD) and theoretically show that if the input distributes as a MMD, the linear discriminant analysis (LDA) classifier will have the best robustness to adversarial examples. We further propose a novel Max-Mahalanobis linear discriminant analysis (MM-LDA) network, which explicitly maps a complicated data distribution in the input space to a MMD in the latent feature space and then applies LDA to make predictions. Our results demonstrate that the MM-LDA networks are significantly more robust to adversarial attacks, and have better performance in class-biased classification.

References in corpus (4)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Ensemble Adversarial Training: Attacks and Defenses
• Adversarial Machine Learning at Scale
• Towards Robust Detection of Adversarial Examples

Cited by in corpus (15)

• On Adaptive Attacks to Adversarial Example Defenses
• Rethinking Softmax Cross-Entropy Loss for Adversarial Robustness
• Boosting Adversarial Training with Hypersphere Embedding
• Improving the Transferability of Adversarial Examples with Resized-Diverse-Inputs, Diversity-Ensemble and Region Fitting
• Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
• Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attacks
• Mixup Inference: Better Exploiting Mixup to Defend Adversarial Attacks
• Cluster Alignment with a Teacher for Unsupervised Domain Adaptation
• Bag of Tricks for Adversarial Training
• Identifying and Resisting Adversarial Videos Using Temporal Consistency
• Unified Classification and Rejection: A One-versus-All Framework
• MiCE: Mixture of Contrastive Experts for Unsupervised Image Clustering
• Improve Adversarial Robustness via Weight Penalization on Classification Layer
• Introducing the DOME Activation Functions
• Generative Max-Mahalanobis Classifiers for Image Classification, Generation and More