3D Self-Supervised Methods for Medical Imaging

Abstract

Self-supervised learning methods have witnessed a recent surge of interest after proving successful in multiple application fields. In this work, we leverage these techniques, and we propose 3D versions for five different self-supervised methods, in the form of proxy tasks. Our methods facilitate neural network feature learning from unlabeled 3D images, aiming to reduce the required cost for expert annotation. The developed algorithms are 3D Contrastive Predictive Coding, 3D Rotation prediction, 3D Jigsaw puzzles, Relative 3D patch location, and 3D Exemplar networks. Our experiments show that pretraining models with our 3D tasks yields more powerful semantic representations, and enables solving downstream tasks more accurately and efficiently, compared to training the models from scratch and to pretraining them on 2D slices. We demonstrate the effectiveness of our methods on three downstream tasks from the medical imaging domain: i) Brain Tumor Segmentation from 3D MRI, ii) Pancreas Tumor Segmentation from 3D CT, and iii) Diabetic Retinopathy Detection from 2D Fundus images. In each task, we assess the gains in data-efficiency, performance, and speed of convergence. Interestingly, we also find gains when transferring the learned representations, by our methods, from a large unlabeled 3D corpus to a small downstream-specific dataset. We achieve results competitive to state-of-the-art solutions at a fraction of the computational expense. We publish our implementations for the developed algorithms (both 3D and 2D versions) as an open-source library, in an effort to allow other researchers to apply and extend our methods on their datasets.

Proceedings of NeurIPS 2020 (Please cite the proceedings version). For source code, see https://github.com/HealthML/self-supervised-3d-tasks

References in corpus (12)

• Adam: A Method for Stochastic Optimization
• Efficient Estimation of Word Representations in Vector Space
• A Simple Framework for Contrastive Learning of Visual Representations
• Representation Learning with Contrastive Predictive Coding
• Conditional Image Generation with PixelCNN Decoders
• Pixel Recurrent Neural Networks
• CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning
• Data-Efficient Image Recognition with Contrastive Predictive Coding
• A large annotated medical image dataset for the development and evaluation of segmentation algorithms
• Improving Cytoarchitectonic Segmentation of Human Brain Areas with Self-supervised Siamese Networks
• Self-Supervised Learning for Spinal MRIs
• Pose from Action: Unsupervised Learning of Pose Features based on Motion

Cited by in corpus (15)

• Recent advances and clinical applications of deep learning in medical image analysis
• Self-Supervised Representation Learning: Introduction, Advances and Challenges
• Models Genesis
• Medical Transformer: Universal Brain Encoder for 3D MRI Analysis
• Cervical Optical Coherence Tomography Image Classification Based on Contrastive Self-Supervised Texture Learning
• MambaMIM: Pre-training Mamba with State Space Token Interpolation and its Application to Medical Image Segmentation
• Registration-Guided Deep Learning Image Segmentation for Cone Beam CT-based Online Adaptive Radiotherapy
• How Transferable Are Self-supervised Features in Medical Image Classification Tasks?
• BC-MRI-SEG: A Breast Cancer MRI Tumor Segmentation Benchmark
• Towards Fine-grained Visual Representations by Combining Contrastive Learning with Image Reconstruction and Attention-weighted Pooling
• Revisiting Hidden Representations in Transfer Learning for Medical Imaging
• Contrastive Learning with Temporal Correlated Medical Images: A Case Study using Lung Segmentation in Chest X-Rays
• Semantic-Aware Contrastive Learning for Multi-object Medical Image Segmentation
• SAR: Scale-Aware Restoration Learning for 3D Tumor Segmentation
• Voxel-level Siamese Representation Learning for Abdominal Multi-Organ Segmentation