Learning a Discriminant Latent Space with Neural Discriminant Analysis

Abstract

Discriminative features play an important role in image and object classification and also in other fields of research such as semi-supervised learning, fine-grained classification, out of distribution detection. Inspired by Linear Discriminant Analysis (LDA), we propose an optimization called Neural Discriminant Analysis (NDA) for Deep Convolutional Neural Networks (DCNNs). NDA transforms deep features to become more discriminative and, therefore, improves the performances in various tasks. Our proposed optimization has two primary goals for inter- and intra-class variances. The first one is to minimize variances within each individual class. The second goal is to maximize pairwise distances between features coming from different classes. We evaluate our NDA optimization in different research fields: general supervised classification, fine-grained classification, semi-supervised learning, and out of distribution detection. We achieve performance improvements in all the fields compared to baseline methods that do not use NDA. Besides, using NDA, we also surpass the state of the art on the four tasks on various testing datasets.

References in corpus (14)

• Prototypical Networks for Few-shot Learning
• FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
• Wide Residual Networks
• Unsupervised Data Augmentation for Consistency Training
• Weight Uncertainty in Neural Networks
• Fine-Grained Visual Classification of Aircraft
• Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks
• Bird Species Categorization Using Pose Normalized Deep Convolutional Nets
• Object-Part Attention Model for Fine-grained Image Classification
• Self-training with Noisy Student improves ImageNet classification
• A Simple Baseline for Bayesian Uncertainty in Deep Learning
• Fine-grained pose prediction, normalization, and recognition
• One Versus all for deep Neural Network Incertitude (OVNNI) quantification
• TRADI: Tracking deep neural network weight distributions for uncertainty estimation