Label Smoothed Embedding Hypothesis for Out-of-Distribution Detection

Abstract

Detecting out-of-distribution (OOD) examples is critical in many applications. We propose an unsupervised method to detect OOD samples using a $k$-NN density estimate with respect to a classification model's intermediate activations on in-distribution samples. We leverage a recent insight about label smoothing, which we call the \emph{Label Smoothed Embedding Hypothesis}, and show that one of the implications is that the $k$-NN density estimator performs better as an OOD detection method both theoretically and empirically when the model is trained with label smoothing. Finally, we show that our proposal outperforms many OOD baselines and also provide new finite-sample high-probability statistical results for $k$-NN density estimation's ability to detect OOD examples.

References in corpus (9)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• When Does Label Smoothing Help?
• Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
• Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks
• Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning
• Outlier detection for patient monitoring and alerting
• Towards better decoding and language model integration in sequence to sequence models
• Uncertainty Sets for Image Classifiers using Conformal Prediction
• An Investigation of how Label Smoothing Affects Generalization