OOD-SEG: Exploiting out-of-distribution detection techniques for learning image segmentation from sparse multi-class positive-only annotations

Abstract

Despite significant advancements, segmentation based on deep neural networks in medical and surgical imaging faces several challenges, two of which we aim to address in this work. First, acquiring complete pixel-level segmentation labels for medical images is time-consuming and requires domain expertise. Second, typical segmentation pipelines cannot detect out-of-distribution (OOD) pixels, leaving them prone to spurious outputs during deployment. In this work, we propose a novel segmentation approach which broadly falls within the positive-unlabelled (PU) learning paradigm and exploits tools from OOD detection techniques. Our framework learns only from sparsely annotated pixels from multiple positive-only classes and does not use any annotation for the background class. These multi-class positive annotations naturally fall within the in-distribution (ID) set. Unlabelled pixels may contain positive classes but also negative ones, including what is typically referred to as \emph{background} in standard segmentation formulations. To the best of our knowledge, this work is the first to formulate multi-class segmentation with sparse positive-only annotations as a pixel-wise PU learning problem and to address it using OOD detection techniques. Here, we forgo the need for background annotation and consider these together with any other unseen classes as part of the OOD set. Our framework can integrate, at a pixel-level, any OOD detection approaches designed for classification tasks. To address the lack of existing OOD datasets and established evaluation metric for medical image segmentation, we propose a cross-validation strategy that treats held-out labelled classes as OOD. Extensive experiments on both multi-class hyperspectral and RGB surgical imaging datasets demonstrate the robustness and generalisation capability of our proposed framework.

Accepted in MedIA

References in corpus (17)

• Interactive Medical Image Segmentation using Deep Learning with Image-specific Fine-tuning
• Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks
• Learning from positive and unlabeled data: a survey
• DeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation
• A Spectral-Spatial-Dependent Global Learning Framework for Insufficient and Imbalanced Hyperspectral Image Classification
• Spatio-spectral classification of hyperspectral images for brain cancer detection during surgical operations
• One-Class Convolutional Neural Network
• Robust deep learning-based semantic organ segmentation in hyperspectral images
• Trends in deep learning for medical hyperspectral image analysis
• Uncertainty-Aware Organ Classification for Surgical Data Science Applications in Laparoscopy
• Video-rate multispectral imaging in laparoscopic surgery: First-in-human application
• Bounding Box Tightness Prior for Weakly Supervised Image Segmentation
• Label-set Loss Functions for Partial Supervision: Application to Fetal Brain 3D MRI Parcellation
• Inter Extreme Points Geodesics for End-to-End Weakly Supervised Image Segmentation
• Synthetic white balancing for intra-operative hyperspectral imaging
• Spatial gradient consistency for unsupervised learning of hyperspectral demosaicking: Application to surgical imaging
• Average Calibration Error: A Differentiable Loss for Improved Reliability in Image Segmentation

Cited by in corpus (1)

• Label tree semantic losses for rich multi-class medical image segmentation