BED: Bi-Encoder-Based Detectors for Out-of-Distribution Detection

Abstract

This paper introduces a novel method leveraging bi-encoder-based detectors along with a comprehensive study comparing different out-of-distribution (OOD) detection methods in NLP using different feature extractors. The feature extraction stage employs popular methods such as Universal Sentence Encoder (USE), BERT, MPNET, and GLOVE to extract informative representations from textual data. The evaluation is conducted on several datasets, including CLINC150, ROSTD-Coarse, SNIPS, and YELLOW. Performance is assessed using metrics such as F1-Score, MCC, FPR@90, FPR@95, AUPR, an AUROC. The experimental results demonstrate that the proposed bi-encoder-based detectors outperform other methods, both those that require OOD labels in training and those that do not, across all datasets, showing great potential for OOD detection in NLP. The simplicity of the training process and the superior detection performance make them applicable to real-world scenarios. The presented methods and benchmarking metrics serve as a valuable resource for future research in OOD detection, enabling further advancements in this field. The code and implementation details can be found on our GitHub repository: https://github.com/yellowmessenger/ood-detection.

arXiv commentPublished in IEEE: https://ieeexplore.ieee.org/document/10389907

References in corpus (10)

• Prototypical Networks for Few-shot Learning
• A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
• Universal Sentence Encoder
• A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
• Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks
• Generalized Out-of-Distribution Detection: A Survey
• To Trust Or Not To Trust A Classifier
• Deep Open Intent Classification with Adaptive Decision Boundary
• Neural Sentence Embedding using Only In-domain Sentences for Out-of-domain Sentence Detection in Dialog Systems
• A Hybrid Architecture for Out of Domain Intent Detection and Intent Discovery