Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks

Abstract

Despite the state-of-the-art performance for medical image segmentation, deep convolutional neural networks (CNNs) have rarely provided uncertainty estimations regarding their segmentation outputs, e.g., model (epistemic) and image-based (aleatoric) uncertainties. In this work, we analyze these different types of uncertainties for CNN-based 2D and 3D medical image segmentation tasks. We additionally propose a test-time augmentation-based aleatoric uncertainty to analyze the effect of different transformations of the input image on the segmentation output. Test-time augmentation has been previously used to improve segmentation accuracy, yet not been formulated in a consistent mathematical framework. Hence, we also propose a theoretical formulation of test-time augmentation, where a distribution of the prediction is estimated by Monte Carlo simulation with prior distributions of parameters in an image acquisition model that involves image transformations and noise. We compare and combine our proposed aleatoric uncertainty with model uncertainty. Experiments with segmentation of fetal brains and brain tumors from 2D and 3D Magnetic Resonance Images (MRI) showed that 1) the test-time augmentation-based aleatoric uncertainty provides a better uncertainty estimation than calculating the test-time dropout-based model uncertainty alone and helps to reduce overconfident incorrect predictions, and 2) our test-time augmentation outperforms a single-prediction baseline and dropout-based multiple predictions.

13 pages, 8 figures, accepted by NeuroComputing

References in corpus (17)

• TensorFlow: A system for large-scale machine learning
• Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• Fully Convolutional Networks for Semantic Segmentation
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
• Interactive Medical Image Segmentation using Deep Learning with Image-specific Fine-tuning
• Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification
• NiftyNet: a deep-learning platform for medical imaging
• Automatic Brain Tumor Segmentation using Cascaded Anisotropic Convolutional Neural Networks
• On the Compactness, Efficiency, and Representation of 3D Convolutional Networks: Brain Parcellation as a Pretext Task
• MURA: Large Dataset for Abnormality Detection in Musculoskeletal Radiographs
• Bayesian Uncertainty Estimation for Batch Normalized Deep Networks
• Image Classification of Melanoma, Nevus and Seborrheic Keratosis by Deep Neural Network Ensemble
• Cascaded V-Net using ROI masks for brain tumor segmentation
• Real-Time Automatic Fetal Brain Extraction in Fetal MRI by Deep Learning
• Data Distillation: Towards Omni-Supervised Learning
• Learning under Distributed Weak Supervision

Cited by in corpus (94)

• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• Recent advances and clinical applications of deep learning in medical image analysis
• TorchIO: A Python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning
• A Survey on Active Learning and Human-in-the-Loop Deep Learning for Medical Image Analysis
• Automatic Brain Tumor Segmentation using Convolutional Neural Networks with Test-Time Augmentation
• Uncertainty-Informed Deep Learning Models Enable High-Confidence Predictions for Digital Histopathology
• MONAI Label: A framework for AI-assisted Interactive Labeling of 3D Medical Images
• Diminishing Uncertainty within the Training Pool: Active Learning for Medical Image Segmentation
• UncertaintyFuseNet: Robust Uncertainty-aware Hierarchical Feature Fusion Model with Ensemble Monte Carlo Dropout for COVID-19 Detection
• A Survey on Epistemic (Model) Uncertainty in Supervised Learning: Recent Advances and Applications
• Semi-Supervised Segmentation of Radiation-Induced Pulmonary Fibrosis from Lung CT Scans with Multi-Scale Guided Dense Attention
• Stochastic Segmentation Networks: Modelling Spatially Correlated Aleatoric Uncertainty
• Integrating uncertainty in deep neural networks for MRI based stroke analysis
• Generalized Wasserstein Dice Score, Distributionally Robust Deep Learning, and Ranger for brain tumor segmentation: BraTS 2020 challenge
• Parameters Estimation for the Cosmic Microwave Background with Bayesian Neural Networks
• Estimating Uncertainty in Neural Networks for Cardiac MRI Segmentation: A Benchmark Study
• Look Before You Leap: An Exploratory Study of Uncertainty Measurement for Large Language Models
• Inter- and intra-uncertainty based feature aggregation model for semi-supervised histopathology image segmentation
• Weakly Supervised Vessel Segmentation in X-ray Angiograms by Self-Paced Learning from Noisy Labels with Suggestive Annotation
• Greedy Policy Search: A Simple Baseline for Learnable Test-Time Augmentation
• Learning Loss for Test-Time Augmentation
• Understanding Calibration of Deep Neural Networks for Medical Image Classification
• Uncertainty Aware Training to Improve Deep Learning Model Calibration for Classification of Cardiac MR Images
• An original framework for Wheat Head Detection using Deep, Semi-supervised and Ensemble Learning within Global Wheat Head Detection (GWHD) Dataset
• Uncertainty-Guided Efficient Interactive Refinement of Fetal Brain Segmentation from Stacks of MRI Slices
• Slice-by-slice deep learning aided oropharyngeal cancer segmentation with adaptive thresholding for spatial uncertainty on FDG PET and CT images
• Task Decomposition and Synchronization for Semantic Biomedical Image Segmentation
• Enhanced Cross-Dataset Electroencephalogram-based Emotion Recognition using Unsupervised Domain Adaptation
• Efficient Semi-Supervised Gross Target Volume of Nasopharyngeal Carcinoma Segmentation via Uncertainty Rectified Pyramid Consistency
• Accuracy, Uncertainty, and Adaptability of Automatic Myocardial ASL Segmentation using Deep CNN
• CertainNet: Sampling-free Uncertainty Estimation for Object Detection
• TMS-Net: A Segmentation Network Coupled With A Run-time Quality Control Method For Robust Cardiac Image Segmentation
• Automatic characterization of boulders on planetary surfaces from high-resolution satellite images
• Self-supervised Learning for Panoptic Segmentation of Multiple Fruit Flower Species
• Temporal Representation Learning on Monocular Videos for 3D Human Pose Estimation
• From Text to Mask: Localizing Entities Using the Attention of Text-to-Image Diffusion Models
• Generalized Wasserstein Dice Loss, Test-time Augmentation, and Transformers for the BraTS 2021 challenge
• A Quantitative Comparison of Epistemic Uncertainty Maps Applied to Multi-Class Segmentation
• Real-time CNN-based Segmentation Architecture for Ball Detection in a Single View Setup
• Embedding-based Instance Segmentation in Microscopy
• Uncertainty-Aware Segmentation Quality Prediction via Deep Learning Bayesian Modeling: Comprehensive Evaluation and Interpretation on Skin Cancer and Liver Segmentation
• Influence of uncertainty estimation techniques on false-positive reduction in liver lesion detection
• Harmonizing Flows: Leveraging normalizing flows for unsupervised and source-free MRI harmonization
• $F$, $B$, Alpha Matting
• Unified Uncertainty Estimation for Cognitive Diagnosis Models
• Few-shot brain segmentation from weakly labeled data with deep heteroscedastic multi-task networks
• Exploiting Uncertainties from Ensemble Learners to Improve Decision-Making in Healthcare AI
• Cycle Consistency-based Uncertainty Quantification of Neural Networks in Inverse Imaging Problems
• Detection of AI Generated Images Using Combined Uncertainty Measures and Particle Swarm Optimised Rejection Mechanism
• CaseNet: Content-Adaptive Scale Interaction Networks for Scene Parsing
• The 2ST-UNet for Pneumothorax Segmentation in Chest X-Rays using ResNet34 as a Backbone for U-Net
• Impact of individual rater style on deep learning uncertainty in medical imaging segmentation
• A Decoupled Uncertainty Model for MRI Segmentation Quality Estimation
• Automatic Segmentation of Organs-at-Risk from Head-and-Neck CT using Separable Convolutional Neural Network with Hard-Region-Weighted Loss
• Boosting Anomaly Detection Using Unsupervised Diverse Test-Time Augmentation
• Propagation and Attribution of Uncertainty in Medical Imaging Pipelines
• Trustworthy AI in Digital Health: A Comprehensive Review of Robustness and Explainability
• UNISELF: A Unified Network with Instance Normalization and Self-Ensembled Lesion Fusion for Multiple Sclerosis Lesion Segmentation
• Uncertainty estimates for semantic segmentation: providing enhanced reliability for automated motor claims handling
• Effect of latent space distribution on the segmentation of images with multiple annotations
• Partial annotations for the segmentation of large structures with low annotation cost
• SkiNet: A Deep Learning Solution for Skin Lesion Diagnosis with Uncertainty Estimation and Explainability
• Uncertainty Estimation in Medical Image Localization: Towards Robust Anterior Thalamus Targeting for Deep Brain Stimulation
• Temporal Uncertainty Localization to Enable Human-in-the-loop Analysis of Dynamic Contrast-enhanced Cardiac MRI Datasets
• Automatic CT Segmentation from Bounding Box Annotations using Convolutional Neural Networks
• Uncertainty categories in medical image segmentation: a study of source-related diversity
• Improving Uncertainty-Error Correspondence in Deep Bayesian Medical Image Segmentation
• Uncertainties of Satellite-based Essential Climate Variables from Deep Learning
• Medical Matting: A New Perspective on Medical Segmentation with Uncertainty
• OOD-SEG: Exploiting out-of-distribution detection techniques for learning image segmentation from sparse multi-class positive-only annotations
• Investigating and Improving Latent Density Segmentation Models for Aleatoric Uncertainty Quantification in Medical Imaging
• Uncertainty Estimation in Deep 2D Echocardiography Segmentation
• Test-time generative augmentation for medical image segmentation
• Deep-learning in the bioimaging wild: Handling ambiguous data with deepflash2
• Automated Labeling of Intracranial Arteries with Uncertainty Quantification Using Deep Learning
• Collaborative Uncertainty in Multi-Agent Trajectory Forecasting
• Cell Tracking according to Biological Needs -- Strong Mitosis-aware Multi-Hypothesis Tracker with Aleatoric Uncertainty
• Improving Calibration and Out-of-Distribution Detection in Medical Image Segmentation with Convolutional Neural Networks
• Uncertainty quantification for White Matter Hyperintensity segmentation detects silent failures and improves automated Fazekas quantification
• Uncertainty-aware Generalized Adaptive CycleGAN
• Medical Image Segmentation with Limited Supervision: A Review of Deep Network Models
• Exploring Instance-Level Uncertainty for Medical Detection
• Diagnostic Uncertainty Calibration: Towards Reliable Machine Predictions in Medical Domain
• Confidence Contours: Uncertainty-Aware Annotation for Medical Semantic Segmentation
• Structured DropConnect for Uncertainty Inference in Image Classification
• Uncertainty-Supervised Interpretable and Robust Evidential Segmentation
• Automatic Segmentation of Gross Target Volume of Nasopharynx Cancer using Ensemble of Multiscale Deep Neural Networks with Spatial Attention
• CUAB: Convolutional Uncertainty Attention Block Enhanced the Chest X-ray Image Analysis
• Robustness via Uncertainty-aware Cycle Consistency
• Toward High-Throughput Artificial Intelligence-Based Segmentation in Oncological PET Imaging
• Partial supervision for the FeTA challenge 2021
• Unsupervised Approaches for Out-Of-Distribution Dermoscopic Lesion Detection
• Bayesian deep learning of affordances from RGB images
• Towards Lower-Dose PET using Physics-Based Uncertainty-Aware Multimodal Learning with Robustness to Out-of-Distribution Data