A Unified Framework for Generalized Low-Shot Medical Image Segmentation with Scarce Data

Abstract

Medical image segmentation has achieved remarkable advancements using deep neural networks (DNNs). However, DNNs often need big amounts of data and annotations for training, both of which can be difficult and costly to obtain. In this work, we propose a unified framework for generalized low-shot (one- and few-shot) medical image segmentation based on distance metric learning (DML). Unlike most existing methods which only deal with the lack of annotations while assuming abundance of data, our framework works with extreme scarcity of both, which is ideal for rare diseases. Via DML, the framework learns a multimodal mixture representation for each category, and performs dense predictions based on cosine distances between the pixels' deep embeddings and the category representations. The multimodal representations effectively utilize the inter-subject similarities and intraclass variations to overcome overfitting due to extremely limited data. In addition, we propose adaptive mixing coefficients for the multimodal mixture distributions to adaptively emphasize the modes better suited to the current input. The representations are implicitly embedded as weights of the fc layer, such that the cosine distances can be computed efficiently via forward propagation. In our experiments on brain MRI and abdominal CT datasets, the proposed framework achieves superior performances for low-shot segmentation towards standard DNN-based (3D U-Net) and classical registration-based (ANTs) methods, e.g., achieving mean Dice coefficients of 81%/69% for brain tissue/abdominal multiorgan segmentation using a single training sample, as compared to 52%/31% and 72%/35% by the U-Net and ANTs, respectively.

Published in IEEE TRANSACTIONS ON MEDICAL IMAGING

References in corpus (11)

• Adam: A Method for Stochastic Optimization
• A Survey on Deep Learning in Medical Image Analysis
• Conditional Generative Adversarial Nets
• Rethinking Atrous Convolution for Semantic Image Segmentation
• Prototypical Networks for Few-shot Learning
• Fully Convolutional Networks for Semantic Segmentation
• Squeeze-and-Excitation Networks
• VoxelMorph: A Learning Framework for Deformable Medical Image Registration
• Evaluate the Malignancy of Pulmonary Nodules Using the 3D Deep Leaky Noisy-or Network
• Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentation
• Few-shot 3D Multi-modal Medical Image Segmentation using Generative Adversarial Learning

Cited by in corpus (4)

• One shot PACS: Patient specific Anatomic Context and Shape prior aware recurrent registration-segmentation of longitudinal thoracic cone beam CTs
• Federated Modality-specific Encoders and Partially Personalized Fusion Decoder for Multimodal Brain Tumor Segmentation
• Multi-Anchor Active Domain Adaptation for Semantic Segmentation
• COIN: Counterfactual inpainting for weakly supervised semantic segmentation for medical images