Med3D: Transfer Learning for 3D Medical Image Analysis

Abstract

The performance on deep learning is significantly affected by volume of training data. Models pre-trained from massive dataset such as ImageNet become a powerful weapon for speeding up training convergence and improving accuracy. Similarly, models based on large dataset are important for the development of deep learning in 3D medical images. However, it is extremely challenging to build a sufficiently large dataset due to difficulty of data acquisition and annotation in 3D medical imaging. We aggregate the dataset from several medical challenges to build 3DSeg-8 dataset with diverse modalities, target organs, and pathologies. To extract general medical three-dimension (3D) features, we design a heterogeneous 3D network called Med3D to co-train multi-domain 3DSeg-8 so as to make a series of pre-trained models. We transfer Med3D pre-trained models to lung segmentation in LIDC dataset, pulmonary nodule classification in LIDC dataset and liver segmentation on LiTS challenge. Experiments show that the Med3D can accelerate the training convergence speed of target 3D medical tasks 2 times compared with model pre-trained on Kinetics dataset, and 10 times compared with training from scratch as well as improve accuracy ranging from 3% to 20%. Transferring our Med3D model on state-the-of-art DenseASPP segmentation network, in case of single model, we achieve 94.6\% Dice coefficient which approaches the result of top-ranged algorithms on the LiTS challenge.

References in corpus (3)

• Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?
• The Kinetics Human Action Video Dataset
• Evaluation of Algorithms for Multi-Modality Whole Heart Segmentation: An Open-Access Grand Challenge

Cited by in corpus (13)

• Free-form tumor synthesis in computed tomography images via richer generative adversarial network
• Supervised Transfer Learning at Scale for Medical Imaging
• Transfer Learning with Edge Attention for Prostate MRI Segmentation
• DoDNet: Learning to segment multi-organ and tumors from multiple partially labeled datasets
• Learning Semantics-enriched Representation via Self-discovery, Self-classification, and Self-restoration
• Revisiting Rubik's Cube: Self-supervised Learning with Volume-wise Transformation for 3D Medical Image Segmentation
• Automated lung segmentation from CT images of normal and COVID-19 pneumonia patients
• Multi-Class 3D Object Detection Within Volumetric 3D Computed Tomography Baggage Security Screening Imagery
• Transferable Visual Words: Exploiting the Semantics of Anatomical Patterns for Self-supervised Learning
• Learning Non-Unique Segmentation with Reward-Penalty Dice Loss
• Multi-task Semi-supervised Learning for Pulmonary Lobe Segmentation
• DeepPrognosis: Preoperative Prediction of Pancreatic Cancer Survival and Surgical Margin via Contrast-Enhanced CT Imaging
• Revisiting 3D Context Modeling with Supervised Pre-training for Universal Lesion Detection in CT Slices