Rainproof: An Umbrella To Shield Text Generators From Out-Of-Distribution Data

Abstract

Implementing effective control mechanisms to ensure the proper functioning and security of deployed NLP models, from translation to chatbots, is essential. A key ingredient to ensure safe system behaviour is Out-Of-Distribution (OOD) detection, which aims to detect whether an input sample is statistically far from the training distribution. Although OOD detection is a widely covered topic in classification tasks, most methods rely on hidden features output by the encoder. In this work, we focus on leveraging soft-probabilities in a black-box framework, i.e. we can access the soft-predictions but not the internal states of the model. Our contributions include: (i) RAINPROOF a Relative informAItioN Projection OOD detection framework; and (ii) a more operational evaluation setting for OOD detection. Surprisingly, we find that OOD detection is not necessarily aligned with task-specific measures. The OOD detector may filter out samples well processed by the model and keep samples that are not, leading to weaker performance. Our results show that RAINPROOF provides OOD detection methods more aligned with task-specific performance metrics than traditional OOD detectors.

References in corpus (18)

• BERTScore: Evaluating Text Generation with BERT
• A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
• PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization
• A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
• DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset
• Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks
• Energy-based Out-of-distribution Detection
• A Simple Fix to Mahalanobis Distance for Improving Near-OOD Detection
• Contrastive Out-of-Distribution Detection for Pretrained Transformers
• Out-of-distribution Detection in Classifiers via Generation
• Feature Space Singularity for Out-of-Distribution Detection
• Adversarial Robustness via Fisher-Rao Regularization
• How to Exploit Hyperspherical Embeddings for Out-of-Distribution Detection?
• Wat zei je? Detecting Out-of-Distribution Translations with Variational Transformers
• Igeood: An Information Geometry Approach to Out-of-Distribution Detection
• Beyond Mahalanobis-Based Scores for Textual OOD Detection
• Open-world Machine Learning: Applications, Challenges, and Opportunities
• Effective Out-of-Distribution Detection in Classifier Based on PEDCC-Loss

Cited by in corpus (1)

• Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph