Impact of individual rater style on deep learning uncertainty in medical imaging segmentation

Abstract

While multiple studies have explored the relation between inter-rater variability and deep learning model uncertainty in medical segmentation tasks, little is known about the impact of individual rater style. This study quantifies rater style in the form of bias and consistency and explores their impacts when used to train deep learning models. Two multi-rater public datasets were used, consisting of brain multiple sclerosis lesion and spinal cord grey matter segmentation. On both datasets, results show a correlation ($R^2=0.60$ and $0.93$) between rater bias and deep learning uncertainty. The impact of label fusion between raters' annotations on this relationship is also explored, and we show that multi-center consensuses are more effective than single-center consensuses to reduce uncertainty, since rater style is mostly center-specific.

arXiv comment17 pages, 8 figures, in submission at MELBA journal

References in corpus (6)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks
• Disentangling Human Error from the Ground Truth in Segmentation of Medical Images
• Benefits of Linear Conditioning with Metadata for Image Segmentation
• Automatic segmentation of spinal multiple sclerosis lesions: How to generalize across MRI contrasts?

Cited by in corpus (1)

• Generating 3D Bio-Printable Patches Using Wound Segmentation and Reconstruction to Treat Diabetic Foot Ulcers