Extreme Consistency: Overcoming Annotation Scarcity and Domain Shifts

Abstract

Supervised learning has proved effective for medical image analysis. However, it can utilize only the small labeled portion of data; it fails to leverage the large amounts of unlabeled data that is often available in medical image datasets. Supervised models are further handicapped by domain shifts, when the labeled dataset, despite being large enough, fails to cover different protocols or ethnicities. In this paper, we introduce \emph{extreme consistency}, which overcomes the above limitations, by maximally leveraging unlabeled data from the same or a different domain in a teacher-student semi-supervised paradigm. Extreme consistency is the process of sending an extreme transformation of a given image to the student network and then constraining its prediction to be consistent with the teacher network's prediction for the untransformed image. The extreme nature of our consistency loss distinguishes our method from related works that yield suboptimal performance by exercising only mild prediction consistency. Our method is 1) auto-didactic, as it requires no extra expert annotations; 2) versatile, as it handles both domain shift and limited annotation problems; 3) generic, as it is readily applicable to classification, segmentation, and detection tasks; and 4) simple to implement, as it requires no adversarial training. We evaluate our method for the tasks of lesion and retinal vessel segmentation in skin and fundus images. Our experiments demonstrate a significant performance gain over both modern supervised networks and recent semi-supervised models. This performance is attributed to the strong regularization enforced by extreme consistency, which enables the student network to learn how to handle extreme variants of both labeled and unlabeled images. This enhances the network's ability to tackle the inevitable same- and cross-domain data variability during inference.

arXiv commentsubmitted for peer-review on March 17

References in corpus (7)

• FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
• Temporal Ensembling for Semi-Supervised Learning
• Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentation
• CenterNet: Keypoint Triplets for Object Detection
• Semi-Supervised Deep Learning for Fully Convolutional Networks
• Few-shot 3D Multi-modal Medical Image Segmentation using Generative Adversarial Learning
• Transformation Consistent Self-ensembling Model for Semi-supervised Medical Image Segmentation

Cited by in corpus (4)

• Active, Continual Fine Tuning of Convolutional Neural Networks for Reducing Annotation Efforts
• Learning Inductive Attention Guidance for Partially Supervised Pancreatic Ductal Adenocarcinoma Prediction
• A survey on Kornia: an Open Source Differentiable Computer Vision Library for PyTorch
• Medical Image Segmentation with Limited Supervision: A Review of Deep Network Models