Rethinking Feature Distribution for Loss Functions in Image Classification
arXiv:1803.02988
Abstract
We propose a large-margin Gaussian Mixture (L-GM) loss for deep neural networks in classification tasks. Different from the softmax cross-entropy loss, our proposal is established on the assumption that the deep features of the training set follow a Gaussian Mixture distribution. By involving a classification margin and a likelihood regularization, the L-GM loss facilitates both a high classification performance and an accurate modeling of the training feature distribution. As such, the L-GM loss is superior to the softmax loss and its major variants in the sense that besides classification, it can be readily used to distinguish abnormal inputs, such as the adversarial examples, based on their features' likelihood to the training feature distribution. Extensive experiments on various recognition benchmarks like MNIST, CIFAR, ImageNet and LFW, as well as on adversarial examples demonstrate the effectiveness of our proposal.
arXiv commentAccepted to CVPR 2018 as spotlight
References in corpus (14)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Distilling the Knowledge in a Neural Network
- Improving neural networks by preventing co-adaptation of feature detectors
- Deep Residual Learning for Image Recognition
- Caffe: Convolutional Architecture for Fast Feature Embedding
- Densely Connected Convolutional Networks
- Wide Residual Networks
- Learning Face Representation from Scratch
- Understanding deep learning requires rethinking generalization
- FractalNet: Ultra-Deep Neural Networks without Residuals
- Large-Margin Softmax Loss for Convolutional Neural Networks
- Deep Learning Face Attributes in the Wild
- Deeply learned face representations are sparse, selective, and robust
- Noisy Softmax: Improving the Generalization Ability of DCNN via Postponing the Early Softmax Saturation