Soft Labeling Affects Out-of-Distribution Detection of Deep Neural Networks
arXiv:2007.03212
Abstract
Soft labeling becomes a common output regularization for generalization and model compression of deep neural networks. However, the effect of soft labeling on out-of-distribution (OOD) detection, which is an important topic of machine learning safety, is not explored. In this study, we show that soft labeling can determine OOD detection performance. Specifically, how to regularize outputs of incorrect classes by soft labeling can deteriorate or improve OOD detection. Based on the empirical results, we postulate a future work for OOD-robust DNNs: a proper output regularization by soft labeling can construct OOD-robust DNNs without additional training of OOD samples or modifying the models, while improving classification accuracy.
arXiv commentICML'20 Workshop on Uncertainty and Robustness in Deep Learning
References in corpus (7)
- Distilling the Knowledge in a Neural Network
- Wide Residual Networks
- A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
- Deep Anomaly Detection with Outlier Exposure
- Self-training with Noisy Student improves ImageNet classification
- Does label smoothing mitigate label noise?
- Revisiting Knowledge Distillation via Label Smoothing Regularization