Contrastive learning of global and local features for medical image segmentation with limited annotations

Abstract

A key requirement for the success of supervised deep learning is a large labeled dataset - a condition that is difficult to meet in medical image analysis. Self-supervised learning (SSL) can help in this regard by providing a strategy to pre-train a neural network with unlabeled data, followed by fine-tuning for a downstream task with limited annotations. Contrastive learning, a particular variant of SSL, is a powerful technique for learning image-level representations. In this work, we propose strategies for extending the contrastive learning framework for segmentation of volumetric medical images in the semi-supervised setting with limited annotations, by leveraging domain-specific and problem-specific cues. Specifically, we propose (1) novel contrasting strategies that leverage structural similarity across volumetric medical images (domain-specific cue) and (2) a local version of the contrastive loss to learn distinctive representations of local regions that are useful for per-pixel segmentation (problem-specific cue). We carry out an extensive evaluation on three Magnetic Resonance Imaging (MRI) datasets. In the limited annotation setting, the proposed method yields substantial improvements compared to other self-supervision and semi-supervised learning techniques. When combined with a simple data augmentation technique, the proposed method reaches within 8% of benchmark performance using only two labeled MRI volumes for training, corresponding to only 4% (for ACDC) of the training data used to train the benchmark. The code is made public at https://github.com/krishnabits001/domain_specific_cl.

arXiv comment18 pages, 2 figures, 10 tables. This article has been accepted as Oral Presentation at NeurIPS 2020 (34th Conference on Neural Information Processing Systems)

Cited by in corpus (12)

• Recent advances and clinical applications of deep learning in medical image analysis
• SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retraining
• A Survey on Deep Learning-based Architectures for Semantic Segmentation on 2D images
• CLIP in Medical Imaging: A Survey
• Reference-guided Pseudo-Label Generation for Medical Semantic Segmentation
• A Review of Predictive and Contrastive Self-supervised Learning for Medical Images
• Federated Contrastive Learning for Volumetric Medical Image Segmentation
• Self-supervised Contrastive Learning of Multi-view Facial Expressions
• MMGL: Multi-Scale Multi-View Global-Local Contrastive learning for Semi-supervised Cardiac Image Segmentation
• Boosting Semi-supervised Image Segmentation with Global and Local Mutual Information Regularization
• Self-Supervised Learning for Medical Image Data with Anatomy-Oriented Imaging Planes
• Learning Generalized Hybrid Proximity Representation for Image Recognition