Confidence Calibration and Predictive Uncertainty Estimation for Deep Medical Image Segmentation

Abstract

Fully convolutional neural networks (FCNs), and in particular U-Nets, have achieved state-of-the-art results in semantic segmentation for numerous medical imaging applications. Moreover, batch normalization and Dice loss have been used successfully to stabilize and accelerate training. However, these networks are poorly calibrated i.e. they tend to produce overconfident predictions both in correct and erroneous classifications, making them unreliable and hard to interpret. In this paper, we study predictive uncertainty estimation in FCNs for medical image segmentation. We make the following contributions: 1) We systematically compare cross entropy loss with Dice loss in terms of segmentation quality and uncertainty estimation of FCNs; 2) We propose model ensembling for confidence calibration of the FCNs trained with batch normalization and Dice loss; 3) We assess the ability of calibrated FCNs to predict segmentation quality of structures and detect out-of-distribution test examples. We conduct extensive experiments across three medical image segmentation applications of the brain, the heart, and the prostate to evaluate our contributions. The results of this study offer considerable insight into the predictive uncertainty estimation and out-of-distribution detection in medical image segmentation and provide practical recipes for confidence calibration. Moreover, we consistently demonstrate that model ensembling improves confidence calibration.

Journal of IEEE Transactions on Medical Imaging

References in corpus (5)

• A Survey on Deep Learning in Medical Image Analysis
• Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations
• Standardized Assessment of Automatic Segmentation of White Matter Hyperintensities and Results of the WMH Segmentation Challenge
• Transfer Learning for Domain Adaptation in MRI: Application in Brain Lesion Segmentation
• Towards increased trustworthiness of deep learning segmentation methods on cardiac MRI

Cited by in corpus (25)

• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• U-Net and its variants for medical image segmentation: theory and applications
• Recent advances and clinical applications of deep learning in medical image analysis
• TransMorph: Transformer for unsupervised medical image registration
• Estimating Uncertainty in Neural Networks for Cardiac MRI Segmentation: A Benchmark Study
• Application of belief functions to medical image segmentation: A review
• Understanding Calibration of Deep Neural Networks for Medical Image Classification
• Uncertainty Aware Training to Improve Deep Learning Model Calibration for Classification of Cardiac MR Images
• Efficient Bayesian Uncertainty Estimation for nnU-Net
• Uncertainty Estimation for Heatmap-based Landmark Localization
• Confidence Calibration for Object Detection and Segmentation
• A Deep Registration Method for Accurate Quantification of Joint Space Narrowing Progression in Rheumatoid Arthritis
• Explaining Predictive Uncertainty by Exposing Second-Order Effects
• A Unified Benchmark for the Unknown Detection Capability of Deep Neural Networks
• Closer Look at the Uncertainty Estimation in Semantic Segmentation under Distributional Shift
• Vision-Based Uncertainty-Aware Motion Planning based on Probabilistic Semantic Segmentation
• Average Calibration Error: A Differentiable Loss for Improved Reliability in Image Segmentation
• Uncertainty estimates for semantic segmentation: providing enhanced reliability for automated motor claims handling
• Temporal Uncertainty Localization to Enable Human-in-the-loop Analysis of Dynamic Contrast-enhanced Cardiac MRI Datasets
• Uncertainty categories in medical image segmentation: a study of source-related diversity
• h-calibration: Rethinking Classifier Recalibration with Probabilistic Error-Bounded Objective
• COVID-19 Detection Using Segmentation, Region Extraction and Classification Pipeline
• Assessing the risk of recurrence in early-stage breast cancer through H&E stained whole slide images
• DA-Cal: Towards Cross-Domain Calibration in Semantic Segmentation
• Uncertainty-Supervised Interpretable and Robust Evidential Segmentation