Ensemble Soft-Margin Softmax Loss for Image Classification
arXiv:1805.03922
Abstract
Softmax loss is arguably one of the most popular losses to train CNN models for image classification. However, recent works have exposed its limitation on feature discriminability. This paper casts a new viewpoint on the weakness of softmax loss. On the one hand, the CNN features learned using the softmax loss are often inadequately discriminative. We hence introduce a soft-margin softmax function to explicitly encourage the discrimination between different classes. On the other hand, the learned classifier of softmax loss is weak. We propose to assemble multiple these weak classifiers to a strong one, inspired by the recognition that the diversity among weak classifiers is critical to a good ensemble. To achieve the diversity, we adopt the Hilbert-Schmidt Independence Criterion (HSIC). Considering these two aspects in one framework, we design a novel loss, named as Ensemble soft-Margin Softmax (EM-Softmax). Extensive experiments on benchmark datasets are conducted to show the superiority of our design over the baseline softmax loss and several state-of-the-art alternatives.
arXiv commentAccepted by IJCAI 2018
References in corpus (12)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Improving neural networks by preventing co-adaptation of feature detectors
- Caffe: Convolutional Architecture for Fast Feature Embedding
- R-FCN: Object Detection via Region-based Fully Convolutional Networks
- Deep Learning using Linear Support Vector Machines
- From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classification
- Large-Margin Softmax Loss for Convolutional Neural Networks
- SphereFace: Deep Hypersphere Embedding for Face Recognition
- A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets
- Learning Deep Features via Congenerous Cosine Loss for Person Recognition
- Range Loss for Deep Face Recognition with Long-tail
- Exclusivity Regularized Machine