Few-shot 3D Multi-modal Medical Image Segmentation using Generative Adversarial Learning

Abstract

We address the problem of segmenting 3D multi-modal medical images in scenarios where very few labeled examples are available for training. Leveraging the recent success of adversarial learning for semi-supervised segmentation, we propose a novel method based on Generative Adversarial Networks (GANs) to train a segmentation model with both labeled and unlabeled images. The proposed method prevents over-fitting by learning to discriminate between true and fake patches obtained by a generator network. Our work extends current adversarial learning approaches, which focus on 2D single-modality images, to the more challenging context of 3D volumes of multiple modalities. The proposed method is evaluated on the problem of segmenting brain MRI from the iSEG-2017 and MRBrainS 2013 datasets. Significant performance improvement is reported, compared to state-of-art segmentation networks trained in a fully-supervised manner. In addition, our work presents a comprehensive analysis of different GAN architectures for semi-supervised segmentation, showing recent techniques like feature matching to yield a higher performance than conventional adversarial training approaches. Our code is publicly available at https://github.com/arnab39/FewShot_GAN-Unet3D

arXiv commentsubmitted to Medical Image Analysis for review

References in corpus (6)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• A Survey on Deep Learning in Medical Image Analysis
• 3D fully convolutional networks for subcortical segmentation in MRI: A large-scale study
• Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation
• Constrained-CNN losses for weakly supervised segmentation
• BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation

Cited by in corpus (27)

• Generative Adversarial Network in Medical Imaging: A Review
• Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentation
• PPGAN: Privacy-preserving Generative Adversarial Network
• Curriculum semi-supervised segmentation
• Self-paced and self-consistent co-training for semi-supervised image segmentation
• Semi-supervised few-shot learning for medical image segmentation
• A Unified Framework for Generalized Low-Shot Medical Image Segmentation with Scarce Data
• Self-Supervision with Superpixels: Training Few-shot Medical Image Segmentation without Annotation
• Exploring Separable Attention for Multi-Contrast MR Image Super-Resolution
• Multi-organ segmentation: a progressive exploration of learning paradigms under scarce annotation
• Medical Image Generation using Generative Adversarial Networks
• 3D Brain and Heart Volume Generative Models: A Survey
• Double-Uncertainty Weighted Method for Semi-supervised Learning
• Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalization
• Expert Training: Task Hardness Aware Meta-Learning for Few-Shot Classification
• Advances in Medical Image Segmentation: A Comprehensive Survey with a Focus on Lumbar Spine Applications
• Extreme Consistency: Overcoming Annotation Scarcity and Domain Shifts
• Deep Learning in Multi-organ Segmentation
• Feeding the zombies: Synthesizing brain volumes using a 3D progressive growing GAN
• Difficulty-aware Meta-learning for Rare Disease Diagnosis
• Few-shot Medical Image Segmentation using a Global Correlation Network with Discriminative Embedding
• Stroke Lesion Segmentation with Visual Cortex Anatomy Alike Neural Nets
• Meta Corrupted Pixels Mining for Medical Image Segmentation
• Improvement of Multiparametric MR Image Segmentation by Augmenting the Data with Generative Adversarial Networks for Glioma Patients
• Learn to Segment Organs with a Few Bounding Boxes
• Belief function-based semi-supervised learning for brain tumor segmentation
• One Representative-Shot Learning Using a Population-Driven Template with Application to Brain Connectivity Classification and Evolution Prediction