A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks

Abstract

We consider the two related problems of detecting if an example is misclassified or out-of-distribution. We present a simple baseline that utilizes probabilities from softmax distributions. Correctly classified examples tend to have greater maximum softmax probabilities than erroneously classified and out-of-distribution examples, allowing for their detection. We assess performance by defining several tasks in computer vision, natural language processing, and automatic speech recognition, showing the effectiveness of this baseline across all. We then show the baseline can sometimes be surpassed, demonstrating the room for future research on these underexplored detection tasks.

arXiv commentPublished as a conference paper at ICLR 2017. 1 Figure in 1 Appendix. Minor changes from the previous version

Cited by in corpus (406)

• On Calibration of Modern Neural Networks
• Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods
• Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
• Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Users
• Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks
• CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features
• AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty
• Energy-based Out-of-distribution Detection
• The Emerging Trends of Multi-Label Learning
• Generalizing to Unseen Domains via Adversarial Data Augmentation
• Predictive Uncertainty Estimation via Prior Networks
• Adversarial Reciprocal Points Learning for Open Set Recognition
• A Comprehensive Survey on Deep Graph Representation Learning
• Generalized Out-of-Distribution Detection: A Survey
• WILDS: A Benchmark of in-the-Wild Distribution Shifts
• Measuring Mathematical Problem Solving With the MATH Dataset
• CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances
• Learning Open Set Network with Discriminative Reciprocal Points
• Galaxy Zoo DECaLS: Detailed Visual Morphology Measurements from Volunteers and Deep Learning for 314,000 Galaxies
• Causal Attention for Interpretable and Generalizable Graph Classification
• Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift
• Natural Adversarial Examples
• An Uncertainty-aware Transfer Learning-based Framework for Covid-19 Diagnosis
• Adversarial Example Detection for DNN Models: A Review and Experimental Comparison
• Scaling Out-of-Distribution Detection for Real-World Settings
• Exploiting Epistemic Uncertainty of Anatomy Segmentation for Anomaly Detection in Retinal OCT
• The Fishyscapes Benchmark: Measuring Blind Spots in Semantic Segmentation
• WAIC, but Why? Generative Ensembles for Robust Anomaly Detection
• Input complexity and out-of-distribution detection with likelihood-based generative models
• Viral Pneumonia Screening on Chest X-ray Images Using Confidence-Aware Anomaly Detection
• Practical Deep Learning with Bayesian Principles
• Scalable agent alignment via reward modeling: a research direction
• RobustBench: a standardized adversarial robustness benchmark
• Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samples
• Zero-Shot Out-of-Distribution Detection Based on the Pre-trained Model CLIP
• Benchmarking Neural Network Robustness to Common Corruptions and Surface Variations
• Addressing Failure Prediction by Learning Model Confidence
• Outlier Exposure with Confidence Control for Out-of-Distribution Detection
• Aligning AI With Shared Human Values
• The Conditional Entropy Bottleneck
• Deep Open Intent Classification with Adaptive Decision Boundary
• Testing for Outliers with Conformal p-values
• How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review
• Improving Reconstruction Autoencoder Out-of-distribution Detection with Mahalanobis Distance
• SegmentMeIfYouCan: A Benchmark for Anomaly Segmentation
• A Simple Fix to Mahalanobis Distance for Improving Near-OOD Detection
• Contrastive Out-of-Distribution Detection for Pretrained Transformers
• Large-Scale Long-Tailed Recognition in an Open World
• Advanced Dropout: A Model-free Methodology for Bayesian Dropout Optimization
• Run-Time Monitoring of Machine Learning for Robotic Perception: A Survey of Emerging Trends
• Metric Learning for Novelty and Anomaly Detection
• Uncertainty in the Variational Information Bottleneck
• TimeMatch: Unsupervised Cross-Region Adaptation by Temporal Shift Estimation
• GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing
• Patch SVDD: Patch-level SVDD for Anomaly Detection and Segmentation
• Graph Communal Contrastive Learning
• Do Deep Generative Models Know What They Don't Know?
• Fine-grained TLS services classification with reject option
• Leveraging Uncertainty Estimates for Predicting Segmentation Quality
• Uncertainty Aware Semi-Supervised Learning on Graph Data
• Contrastive Training for Improved Out-of-Distribution Detection
• Practical Solutions for Machine Learning Safety in Autonomous Vehicles
• Adversarial Examples in Modern Machine Learning: A Review
• Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
• Understanding Softmax Confidence and Uncertainty
• SSD: A Unified Framework for Self-Supervised Outlier Detection
• Discriminative out-of-distribution detection for semantic segmentation
• Deep Weighted Averaging Classifiers
• Why is the Mahalanobis Distance Effective for Anomaly Detection?
• Beyond Robustness: A Taxonomy of Approaches towards Resilient Multi-Robot Systems
• Denoising Autoencoders for Overgeneralization in Neural Networks
• Calibration of Encoder Decoder Models for Neural Machine Translation
• Open-Set Recognition: a Good Closed-Set Classifier is All You Need?
• A Benchmark of Medical Out of Distribution Detection
• FINE Samples for Learning with Noisy Labels
• On the Importance of Gradients for Detecting Distributional Shifts in the Wild
• Unified Probabilistic Deep Continual Learning through Generative Replay and Open Set Recognition
• Innovations in Integrating Machine Learning and Agent-Based Modeling of Biomedical Systems
• Can You Trust This Prediction? Auditing Pointwise Reliability After Learning
• Unsupervised Domain Adaptation of Black-Box Source Models
• Probabilistic Modeling of Deep Features for Out-of-Distribution and Adversarial Detection
• Unsupervised Outlier Detection using Memory and Contrastive Learning
• An Empirical Evaluation on Robustness and Uncertainty of Regularization Methods
• Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Data
• Can multi-label classification networks know what they don't know?
• Detecting Out-of-Distribution Inputs in Deep Neural Networks Using an Early-Layer Output
• Novelty Detection with GAN
• Deep Verifier Networks: Verification of Deep Discriminative Models with Deep Generative Models
• Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networks
• Uncertainty in Gradient Boosting via Ensembles
• Energy-based Detection of Adverse Weather Effects in LiDAR Data
• Uncertainty Estimation in Autoregressive Structured Prediction
• Understanding Anomaly Detection with Deep Invertible Networks through Hierarchies of Distributions and Features
• Robust Out-of-distribution Detection for Neural Networks
• How Does Fine-Tuning Impact Out-of-Distribution Detection for Vision-Language Models?
• Learning on Graphs with Out-of-Distribution Nodes
• Open-set Label Noise Can Improve Robustness Against Inherent Label Noise
• Curriculum learning for improved femur fracture classification: scheduling data with prior knowledge and uncertainty
• Prediction Calibration for Generalized Few-shot Semantic Segmentation
• TEXTOIR: An Integrated and Visualized Platform for Text Open Intent Recognition
• Learning Discriminative Representations and Decision Boundaries for Open Intent Detection
• Entropic Out-of-Distribution Detection: Seamless Detection of Unknown Examples
• Into the Unknown: Active Monitoring of Neural Networks
• Outside the Box: Abstraction-Based Monitoring of Neural Networks
• Improved Contrastive Divergence Training of Energy Based Models
• Pretrained Transformers Improve Out-of-Distribution Robustness
• Laplace Redux -- Effortless Bayesian Deep Learning
• Feature Space Singularity for Out-of-Distribution Detection
• What can AI do for me: Evaluating Machine Learning Interpretations in Cooperative Play
• BERT & Family Eat Word Salad: Experiments with Text Understanding
• Improving colonoscopy lesion classification using semi-supervised deep learning
• Adversarial Learning in Statistical Classification: A Comprehensive Review of Defenses Against Attacks
• Self-Supervised Training Enhances Online Continual Learning
• Uncertainty Based Detection and Relabeling of Noisy Image Labels
• Deep CNN-based Multi-task Learning for Open-Set Recognition
• A semi-supervised Machine Learning search for never-seen Gravitational-Wave sources
• Reliable Neural Networks for Regression Uncertainty Estimation
• Detecting Out-of-Distribution Examples with In-distribution Examples and Gram Matrices
• Inhibited Softmax for Uncertainty Estimation in Neural Networks
• A Comparison of Uncertainty Estimation Approaches in Deep Learning Components for Autonomous Vehicle Applications
• Learning Densities in Feature Space for Reliable Segmentation of Indoor Scenes
• A Unified Survey on Anomaly, Novelty, Open-Set, and Out-of-Distribution Detection: Solutions and Future Challenges
• Discriminative Nearest Neighbor Few-Shot Intent Detection by Transferring Natural Language Inference
• Incorporating Experts' Judgment into Machine Learning Models
• A Less Biased Evaluation of Out-of-distribution Sample Detectors
• Unbiased Mean Teacher for Cross-domain Object Detection
• Multiobjective Evolutionary Pruning of Deep Neural Networks with Transfer Learning for improving their Performance and Robustness
• On Last-Layer Algorithms for Classification: Decoupling Representation from Uncertainty Estimation
• SentiBERT: A Transferable Transformer-Based Architecture for Compositional Sentiment Semantics
• M2m: Imbalanced Classification via Major-to-minor Translation
• Prior and Posterior Networks: A Survey on Evidential Deep Learning Methods For Uncertainty Estimation
• Statistical process monitoring of artificial neural networks
• A graph neural network-based model with Out-of-Distribution Robustness for enhancing Antiretroviral Therapy Outcome Prediction for HIV-1
• Corner Cases for Visual Perception in Automated Driving: Some Guidance on Detection Approaches
• Multimodal Detection of Unknown Objects on Roads for Autonomous Driving
• AutoOD: Automated Outlier Detection via Curiosity-guided Search and Self-imitation Learning
• Trust Issues: Uncertainty Estimation Does Not Enable Reliable OOD Detection On Medical Tabular Data
• ProtoAttend: Attention-Based Prototypical Learning
• An Outlier Exposure Approach to Improve Visual Anomaly Detection Performance for Mobile Robots
• Understanding Open-Set Recognition by Jacobian Norm and Inter-Class Separation
• Deep Deterministic Uncertainty for Semantic Segmentation
• Hyperparameter-Free Out-of-Distribution Detection Using Softmax of Scaled Cosine Similarity
• Tail of Distribution GAN (TailGAN): Generative-Adversarial-Network-Based Boundary Formation
• A Variational Dirichlet Framework for Out-of-Distribution Detection
• Novelty Detection Via Blurring
• Building robust classifiers through generation of confident out of distribution examples
• Detecting Adversarial Examples and Other Misclassifications in Neural Networks by Introspection
• Uncertainty Estimation and Out-of-Distribution Detection for Counterfactual Explanations: Pitfalls and Solutions
• Boundary of Distribution Support Generator (BDSG): Sample Generation on the Boundary
• Robot-Assisted Feeding: Generalizing Skewering Strategies across Food Items on a Realistic Plate
• DraftRec: Personalized Draft Recommendation for Winning in Multi-Player Online Battle Arena Games
• DS-UI: Dual-Supervised Mixture of Gaussian Mixture Models for Uncertainty Inference
• The Hidden Uncertainty in a Neural Networks Activations
• Enhancing Consistency and Mitigating Bias: A Data Replay Approach for Incremental Learning
• Self-Supervised Features Improve Open-World Learning
• Predictive Uncertainty Quantification with Compound Density Networks
• Stable Neural ODE with Lyapunov-Stable Equilibrium Points for Defending Against Adversarial Attacks
• Deep pNML: Predictive Normalized Maximum Likelihood for Deep Neural Networks
• Unified Classification and Rejection: A One-versus-All Framework
• A Unified Benchmark for the Unknown Detection Capability of Deep Neural Networks
• Unsupervised Out-of-Distribution Detection by Maximum Classifier Discrepancy
• Synthesize then Compare: Detecting Failures and Anomalies for Semantic Segmentation
• ATOM: Robustifying Out-of-distribution Detection Using Outlier Mining
• M2IOSR: Maximal Mutual Information Open Set Recognition
• Real-time Out-of-distribution Detection in Learning-Enabled Cyber-Physical Systems
• Dense Uncertainty Estimation
• Fast Entropy-Based Methods of Word-Level Confidence Estimation for End-To-End Automatic Speech Recognition
• FRODO: Free rejection of out-of-distribution samples: application to chest x-ray analysis
• DOCTOR: A Simple Method for Detecting Misclassification Errors
• In Search of netUnicorn: A Data-Collection Platform to Develop Generalizable ML Models for Network Security Problems
• Sketching Curvature for Efficient Out-of-Distribution Detection for Deep Neural Networks
• Revisiting One-vs-All Classifiers for Predictive Uncertainty and Out-of-Distribution Detection in Neural Networks
• Learning Adaptive Embedding Considering Incremental Class
• Accelerating Monte Carlo Bayesian Inference via Approximating Predictive Uncertainty over Simplex
• Online Algorithms for Hierarchical Inference in Deep Learning applications at the Edge
• Unsupervised Domain Adaptation in the Absence of Source Data
• MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Space
• On Mixup Regularization
• Anomalous Example Detection in Deep Learning: A Survey
• Robust Semantic Segmentation with Ladder-DenseNet Models
• No True State-of-the-Art? OOD Detection Methods are Inconsistent across Datasets
• Likelihood Ratios for Out-of-Distribution Detection
• Modeling Uncertainty by Learning a Hierarchy of Deep Neural Connections
• DeepLens: Interactive Out-of-distribution Data Detection in NLP Models
• flexgrid2vec: Learning Efficient Visual Representations Vectors
• Classification-Reconstruction Learning for Open-Set Recognition
• OneFlow: One-class flow for anomaly detection based on a minimal volume region
• Stationary Activations for Uncertainty Calibration in Deep Learning
• Prior choice affects ability of Bayesian neural networks to identify unknowns
• Label-Free Model Evaluation with Semi-Structured Dataset Representations
• Negative Data Augmentation
• Hierarchical VAEs Know What They Don't Know
• A Critical Evaluation of Open-World Machine Learning
• Out-of-domain Detection for Natural Language Understanding in Dialog Systems
• Adaptive Confidence Smoothing for Generalized Zero-Shot Learning
• Open-World Lifelong Graph Learning
• Multiscale Score Matching for Out-of-Distribution Detection
• Double-Adversarial Activation Anomaly Detection: Adversarial Autoencoders are Anomaly Generators
• A Boundary Based Out-of-Distribution Classifier for Generalized Zero-Shot Learning
• TR-BERT: Dynamic Token Reduction for Accelerating BERT Inference
• W2WNet: a two-module probabilistic Convolutional Neural Network with embedded data cleansing functionality
• Reliable and Trustworthy Machine Learning for Health Using Dataset Shift Detection
• Adaptation Across Extreme Variations using Unlabeled Domain Bridges
• DQI: Measuring Data Quality in NLP
• OpenCoS: Contrastive Semi-supervised Learning for Handling Open-set Unlabeled Data
• Calibrated Language Model Fine-Tuning for In- and Out-of-Distribution Data
• Probing the Purview of Neural Networks via Gradient Analysis
• Quizbowl: The Case for Incremental Question Answering
• No MCMC for me: Amortized sampling for fast and stable training of energy-based models
• MetaFusion: Controlled False-Negative Reduction of Minority Classes in Semantic Segmentation
• Sinkhorn Label Allocation: Semi-Supervised Classification via Annealed Self-Training
• DROCC: Deep Robust One-Class Classification
• Extraction of Complex DNN Models: Real Threat or Boogeyman?
• SQUID: Deep Feature In-Painting for Unsupervised Anomaly Detection
• Detecting Out-of-distribution Samples via Variational Auto-encoder with Reliable Uncertainty Estimation
• Domain segmentation and adjustment for generalized zero-shot learning
• DICE: Diversity in Deep Ensembles via Conditional Redundancy Adversarial Estimation
• ReSmooth: Detecting and Utilizing OOD Samples when Training with Data Augmentation
• Robust Semantic Segmentation with Superpixel-Mix
• Do We Really Need to Learn Representations from In-domain Data for Outlier Detection?
• OoDAnalyzer: Interactive Analysis of Out-of-Distribution Samples
• Multi-Task Curriculum Framework for Open-Set Semi-Supervised Learning
• Evaluating and Boosting Uncertainty Quantification in Classification
• Detecting Errors and Estimating Accuracy on Unlabeled Data with Self-training Ensembles
• E-Stitchup: Data Augmentation for Pre-Trained Embeddings
• Mean-Field Approximation to Gaussian-Softmax Integral with Application to Uncertainty Estimation
• Learning Placeholders for Open-Set Recognition
• Regularizing Attention Networks for Anomaly Detection in Visual Question Answering
• Soft Labeling Affects Out-of-Distribution Detection of Deep Neural Networks
• AppealNet: An Efficient and Highly-Accurate Edge/Cloud Collaborative Architecture for DNN Inference
• Deep Learning and Traffic Classification: Lessons learned from a commercial-grade dataset with hundreds of encrypted and zero-day applications
• On the Sample Complexity of Adversarial Multi-Source PAC Learning
• Selective Classification Can Magnify Disparities Across Groups
• Selective Question Answering under Domain Shift
• Few-Shot Open-Set Recognition using Meta-Learning
• Confidence-based Ensembles of End-to-End Speech Recognition Models
• TRIER: Template-Guided Neural Networks for Robust and Interpretable Sleep Stage Identification from EEG Recordings
• InFlow: Robust outlier detection utilizing Normalizing Flows
• Improving Road Segmentation in Challenging Domains Using Similar Place Priors
• Resisting Out-of-Distribution Data Problem in Perturbation of XAI
• Bounds all around: training energy-based models with bidirectional bounds
• Measuring the Novelty of Natural Language Text Using the Conjunctive Clauses of a Tsetlin Machine Text Classifier
• An evaluation of word-level confidence estimation for end-to-end automatic speech recognition
• Practical Evaluation of Out-of-Distribution Detection Methods for Image Classification
• Learning to Predict Trustworthiness with Steep Slope Loss
• Detecting Unknown DGAs without Context Information
• APP: Adaptive Prototypical Pseudo-Labeling for Few-shot OOD Detection
• Perturbation-based Self-supervised Attention for Attention Bias in Text Classification
• Calibrating Deep Neural Network Classifiers on Out-of-Distribution Datasets
• SLURP: Side Learning Uncertainty for Regression Problems
• Provably Robust Detection of Out-of-distribution Data (almost) for free
• URSABench: Comprehensive Benchmarking of Approximate Bayesian Inference Methods for Deep Neural Networks
• Our Evaluation Metric Needs an Update to Encourage Generalization
• Bayesian OOD detection with aleatoric uncertainty and outlier exposure
• Know Your Limits: Uncertainty Estimation with ReLU Classifiers Fails at Reliable OOD Detection
• Modeling Token-level Uncertainty to Learn Unknown Concepts in SLU via Calibrated Dirichlet Prior RNN
• A Simple Framework for Uncertainty in Contrastive Learning
• A t-distribution based operator for enhancing out of distribution robustness of neural network classifiers
• Class Anchor Clustering: a Loss for Distance-based Open Set Recognition
• Out-of-Distribution Detection for Skin Lesion Images with Deep Isolation Forest
• SCOUT: Self-aware Discriminant Counterfactual Explanations
• Learning Global and Local Features of Normal Brain Anatomy for Unsupervised Abnormality Detection
• Active Deep Learning on Entity Resolution by Risk Sampling
• NGC: A Unified Framework for Learning with Open-World Noisy Data
• Increasing Trustworthiness of Deep Neural Networks via Accuracy Monitoring
• Synthetic Unknown Class Learning for Learning Unknowns
• Semi-supervised learning by selective training with pseudo labels via confidence estimation
• A machine learning photon detection algorithm for coherent X-ray ultrafast fluctuation analysis
• Confidence Estimation for Attention-based Sequence-to-sequence Models for Speech Recognition
• Adversarial Robustness Study of Convolutional Neural Network for Lumbar Disk Shape Reconstruction from MR images
• Model selection for deep audio source separation via clustering analysis
• Learning with Out-of-Distribution Data for Audio Classification
• Enhanced Isotropy Maximization Loss: Seamless and High-Performance Out-of-Distribution Detection Simply Replacing the SoftMax Loss
• Are CLIP features all you need for Universal Synthetic Image Origin Attribution?
• Evaluating subgroup disparity using epistemic uncertainty in mammography
• An Algorithm for Out-Of-Distribution Attack to Neural Network Encoder
• Single Layer Predictive Normalized Maximum Likelihood for Out-of-Distribution Detection
• Thinkback: Task-SpecificOut-of-Distribution Detection
• TRADI: Tracking deep neural network weight distributions for uncertainty estimation
• Empirical Frequentist Coverage of Deep Learning Uncertainty Quantification Procedures
• An Infinite-Feature Extension for Bayesian ReLU Nets That Fixes Their Asymptotic Overconfidence
• Likelihood Ratios and Generative Classifiers for Unsupervised Out-of-Domain Detection In Task Oriented Dialog
• Deep Metric Learning for Open World Semantic Segmentation
• OutFlip: Generating Out-of-Domain Samples for Unknown Intent Detection with Natural Language Attack
• Being a Bit Frequentist Improves Bayesian Neural Networks
• Deep Anomaly Detection by Residual Adaptation
• CheXbreak: Misclassification Identification for Deep Learning Models Interpreting Chest X-rays
• CheXseen: Unseen Disease Detection for Deep Learning Interpretation of Chest X-rays
• Random Polytope Descriptors
• Likelihood Assignment for Out-of-Distribution Inputs in Deep Generative Models is Sensitive to Prior Distribution Choice
• Anomaly Detection by Recombining Gated Unsupervised Experts
• Unsupervised Out-of-Domain Detection via Pre-trained Transformers
• Masked Contrastive Learning for Anomaly Detection
• Types of Out-of-Distribution Texts and How to Detect Them
• On Out-of-distribution Detection with Energy-based Models
• Utilizing Network Properties to Detect Erroneous Inputs
• What Does Rotation Prediction Tell Us about Classifier Accuracy under Varying Testing Environments?
• Shifting Transformation Learning for Out-of-Distribution Detection
• Modeling Discriminative Representations for Out-of-Domain Detection with Supervised Contrastive Learning
• Identifying Untrustworthy Predictions in Neural Networks by Geometric Gradient Analysis
• Informed Democracy: Voting-based Novelty Detection for Action Recognition
• Approaching Neural Network Uncertainty Realism
• Adaptive Deep Learning for Entity Resolution by Risk Analysis
• Bridging In- and Out-of-distribution Samples for Their Better Discriminability
• Classification Uncertainty of Deep Neural Networks Based on Gradient Information
• Exploring Vicinal Risk Minimization for Lightweight Out-of-Distribution Detection
• Unsupervised Anomaly Detection From Semantic Similarity Scores
• Uncertainty Estimation and Calibration with Finite-State Probabilistic RNNs
• Rainproof: An Umbrella To Shield Text Generators From Out-Of-Distribution Data
• Mixtures of Laplace Approximations for Improved Post-Hoc Uncertainty in Deep Learning
• Robust Deep Learning Ensemble against Deception
• A statistical framework for efficient out of distribution detection in deep neural networks
• Probabilistic Approach for Road-Users Detection
• Test Sample Accuracy Scales with Training Sample Density in Neural Networks
• Detecting Out-of-distribution Objects Using Neuron Activation Patterns
• Unsupervised Out-of-Distribution Dialect Detection with Mahalanobis Distance
• Deep Out-of-Distribution Uncertainty Quantification via Weight Entropy Maximization
• Out-of-distribution data supervision towards biomedical semantic segmentation
• Requirements for Developing Robust Neural Networks
• Self-supervised Benchmark Lottery on ImageNet: Do Marginal Improvements Translate to Improvements on Similar Datasets?
• Camera Pose Matters: Improving Depth Prediction by Mitigating Pose Distribution Bias
• A Multi-Scale Framework for Out-of-Distribution Detection in Dermoscopic Images
• Online Black-Box Confidence Estimation of Deep Neural Networks
• CheapET-3: Cost-Efficient Use of Remote DNN Models
• Detecting and Understanding Generalization Barriers for Neural Machine Translation
• DeepStreamCE: A Streaming Approach to Concept Evolution Detection in Deep Neural Networks
• One Representation to Rule Them All: Identifying Out-of-Support Examples in Few-shot Learning with Generic Representations
• Conformance Constraint Discovery: Measuring Trust in Data-Driven Systems
• Towards Robust Pattern Recognition: A Review
• Controllable Confidence-Based Image Denoising
• Out-of-Scope Intent Detection with Self-Supervision and Discriminative Training
• Pool of Experts: Realtime Querying Specialized Knowledge in Massive Neural Networks
• Deep Residual Flow for Out of Distribution Detection
• A Tale Of Two Long Tails
• Being Single Has Benefits. Instance Poisoning to Deceive Malware Classifiers
• Uncertainty estimation for classification and risk prediction on medical tabular data
• Calibrated Top-1 Uncertainty estimates for classification by score based models
• Out-Of-Distribution Detection With Subspace Techniques And Probabilistic Modeling Of Features
• Out-distribution aware Self-training in an Open World Setting
• Relevant-features based Auxiliary Cells for Energy Efficient Detection of Natural Errors
• Energy-efficient and Robust Cumulative Training with Net2Net Transformation
• MOOD: Multi-level Out-of-distribution Detection
• The Compact Support Neural Network
• ALT-MAS: A Data-Efficient Framework for Active Testing of Machine Learning Algorithms
• Image-based OoD-Detector Principles on Graph-based Input Data in Human Action Recognition
• Modeling Irregular Astronomical Time Series with Neural Stochastic Delay Differential Equations
• Understanding Classifier Mistakes with Generative Models
• Unsupervised Learning of Multi-level Structures for Anomaly Detection
• Energy-based Unknown Intent Detection with Data Manipulation
• Interpretable BoW Networks for Adversarial Example Detection
• Accurately Classifying Out-Of-Distribution Data in Facial Recognition
• Familiarity-Based Open-Set Recognition Under Adversarial Attacks
• Lyapunov-based uncertainty-aware safe reinforcement learning
• Transductive image segmentation: Self-training and effect of uncertainty estimation
• BED: Bi-Encoder-Based Detectors for Out-of-Distribution Detection
• Why Should we Combine Training and Post-Training Methods for Out-of-Distribution Detection?
• Out-of-Distribution Example Detection in Deep Neural Networks using Distance to Modelled Embedding
• MAGDiff: Covariate Data Set Shift Detection via Activation Graphs of Deep Neural Networks
• Active Learning for UAV-based Semantic Mapping
• GOLD: Improving Out-of-Scope Detection in Dialogues using Data Augmentation
• Towards Interpretable and Learnable Risk Analysis for Entity Resolution
• A novel network training approach for open set image recognition
• DOODLER: Determining Out-Of-Distribution Likelihood from Encoder Reconstructions
• $f$-Cal: Calibrated aleatoric uncertainty estimation from neural networks for robot perception
• Why Out-of-distribution Detection in CNNs Does Not Like Mahalanobis -- and What to Use Instead
• Neural Bootstrapper
• Out-of-Distribution Detection for Dermoscopic Image Classification
• Perceptual Consistency in Video Segmentation
• PnPOOD : Out-Of-Distribution Detection for Text Classification via Plug andPlay Data Augmentation
• Generalization by Recognizing Confusion
• Uncertainty Surrogates for Deep Learning
• Testing for Typicality with Respect to an Ensemble of Learned Distributions
• Sub-clusters of Normal Data for Anomaly Detection
• Incorporating Crowdsourced Annotator Distributions into Ensemble Modeling to Improve Classification Trustworthiness for Ancient Greek Papyri
• Revealing the Distributional Vulnerability of Discriminators by Implicit Generators
• Evaluation of Out-of-Distribution Detection Performance of Self-Supervised Learning in a Controllable Environment
• Open-Set Representation Learning through Combinatorial Embedding
• Does the dataset meet your expectations? Explaining sample representation in image data
• Detection of Dataset Shifts in Learning-Enabled Cyber-Physical Systems using Variational Autoencoder for Regression
• Open-World Class Discovery with Kernel Networks
• Open-set Adversarial Defense
• Ultra-High Dimensional Sparse Representations with Binarization for Efficient Text Retrieval
• Plug-and-Play Anomaly Detection with Expectation Maximization Filtering
• Anomaly-Aware Semantic Segmentation by Leveraging Synthetic-Unknown Data
• Consensus Synergizes with Memory: A Simple Approach for Anomaly Segmentation in Urban Scenes
• Generative Max-Mahalanobis Classifiers for Image Classification, Generation and More
• Uncertainty estimation under model misspecification in neural network regression
• Model Rectification via Unknown Unknowns Extraction from Deployment Samples
• Open-set 3D Object Detection
• Multi-Class Anomaly Detection
• Novel Slot Detection: A Benchmark for Discovering Unknown Slot Types in the Task-Oriented Dialogue System
• Toward Auto-evaluation with Confidence-based Category Relation-aware Regression
• Class-Similarity Based Label Smoothing for Confidence Calibration
• The Robust Semantic Segmentation UNCV2023 Challenge Results
• LLC: Accurate, Multi-purpose Learnt Low-dimensional Binary Codes
• Frustratingly Easy Uncertainty Estimation for Distribution Shift
• Information Theoretic Structured Generative Modeling
• Model-based Interactive Semantic Parsing: A Unified Framework and A Text-to-SQL Case Study
• Self-Checking Deep Neural Networks in Deployment
• Test for non-negligible adverse shifts
• The Effect of Optimization Methods on the Robustness of Out-of-Distribution Detection Approaches
• Kullback-Leibler Divergence-Based Out-of-Distribution Detection with Flow-Based Generative Models
• On the detection of Out-Of-Distribution samples in Multiple Instance Learning
• Joint Distribution across Representation Space for Out-of-Distribution Detection
• Interviewer-Candidate Role Play: Towards Developing Real-World NLP Systems
• Toward a Realistic Benchmark for Out-of-Distribution Detection