Bidirectional RNN-based Few Shot Learning for 3D Medical Image Segmentation

Abstract

Segmentation of organs of interest in 3D medical images is necessary for accurate diagnosis and longitudinal studies. Though recent advances using deep learning have shown success for many segmentation tasks, large datasets are required for high performance and the annotation process is both time consuming and labor intensive. In this paper, we propose a 3D few shot segmentation framework for accurate organ segmentation using limited training samples of the target organ annotation. To achieve this, a U-Net like network is designed to predict segmentation by learning the relationship between 2D slices of support data and a query image, including a bidirectional gated recurrent unit (GRU) that learns consistency of encoded features between adjacent slices. Also, we introduce a transfer learning method to adapt the characteristics of the target image and organ by updating the model before testing with arbitrary support and query data sampled from the support data. We evaluate our proposed model using three 3D CT datasets with annotations of different organs. Our model yielded significantly improved performance over state-of-the-art few shot segmentation models and was comparable to a fully supervised model trained with more target training data.

arXiv commentSubmitted to AAAI21

References in corpus (14)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• Attention U-Net: Learning Where to Look for the Pancreas
• A large annotated medical image dataset for the development and evaluation of segmentation algorithms
• Learning to Propagate Labels: Transductive Propagation Network for Few-shot Learning
• Few-Shot Learning with Graph Neural Networks
• FSS-1000: A 1000-Class Dataset for Few-Shot Segmentation
• Improving Deep Pancreas Segmentation in CT and MRI Images via Recurrent Neural Contextual Learning and Direct Loss Function
• Pancreas Segmentation in CT and MRI Images via Domain Specific Network Designing and Recurrent Neural Contextual Learning
• Part-aware Prototype Network for Few-shot Semantic Segmentation
• A 3D Coarse-to-Fine Framework for Volumetric Medical Image Segmentation
• A Model-Driven Stack-Based Fully Convolutional Network for Pancreas Segmentation