Med3D: Transfer Learning for 3D Medical Image Analysis

Abstract

The performance on deep learning is significantly affected by volume of training data. Models pre-trained from massive dataset such as ImageNet become a powerful weapon for speeding up training convergence and improving accuracy. Similarly, models based on large dataset are important for the development of deep learning in 3D medical images. However, it is extremely challenging to build a sufficiently large dataset due to difficulty of data acquisition and annotation in 3D medical imaging. We aggregate the dataset from several medical challenges to build 3DSeg-8 dataset with diverse modalities, target organs, and pathologies. To extract general medical three-dimension (3D) features, we design a heterogeneous 3D network called Med3D to co-train multi-domain 3DSeg-8 so as to make a series of pre-trained models. We transfer Med3D pre-trained models to lung segmentation in LIDC dataset, pulmonary nodule classification in LIDC dataset and liver segmentation on LiTS challenge. Experiments show that the Med3D can accelerate the training convergence speed of target 3D medical tasks 2 times compared with model pre-trained on Kinetics dataset, and 10 times compared with training from scratch as well as improve accuracy ranging from 3% to 20%. Transferring our Med3D model on state-the-of-art DenseASPP segmentation network, in case of single model, we achieve 94.6\% Dice coefficient which approaches the result of top-ranged algorithms on the LiTS challenge.

References in corpus (5)

• Adam: A Method for Stochastic Optimization
• Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?
• The Kinetics Human Action Video Dataset
• Rethinking ImageNet Pre-training
• Evaluation of Algorithms for Multi-Modality Whole Heart Segmentation: An Open-Access Grand Challenge

Cited by in corpus (47)

• Recent advances and clinical applications of deep learning in medical image analysis
• Models Genesis
• CLIP-Driven Universal Model for Organ Segmentation and Tumor Detection
• A Survey on Incorporating Domain Knowledge into Deep Learning for Medical Image Analysis
• Large AI Models in Health Informatics: Applications, Challenges, and the Future
• Hierarchical Amortized Training for Memory-efficient High Resolution 3D GAN
• Recent Progress in Transformer-based Medical Image Analysis
• Reinventing 2D Convolutions for 3D Images
• Conditional Diffusion Models for Semantic 3D Brain MRI Synthesis
• CLIP in Medical Imaging: A Survey
• Active, Continual Fine Tuning of Convolutional Neural Networks for Reducing Annotation Efforts
• Free-form tumor synthesis in computed tomography images via richer generative adversarial network
• FakeNews: GAN-based generation of realistic 3D volumetric data -- A systematic review and taxonomy
• Medical Transformer: Universal Brain Encoder for 3D MRI Analysis
• sMRI-PatchNet: A novel explainable patch-based deep learning network for Alzheimer's disease diagnosis and discriminative atrophy localisation with Structural MRI
• Supervised Transfer Learning at Scale for Medical Imaging
• Guided Reconstruction with Conditioned Diffusion Models for Unsupervised Anomaly Detection in Brain MRIs
• Conquering Data Variations in Resolution: A Slice-Aware Multi-Branch Decoder Network
• HOPE: Hybrid-granularity Ordinal Prototype Learning for Progression Prediction of Mild Cognitive Impairment
• Weakly-supervised Learning For Catheter Segmentation in 3D Frustum Ultrasound
• Lesion-Aware Cross-Phase Attention Network for Renal Tumor Subtype Classification on Multi-Phase CT Scans
• 3D Brain and Heart Volume Generative Models: A Survey
• Contrastive Registration for Unsupervised Medical Image Segmentation
• Weakly Supervised Attention Model for RV StrainClassification from volumetric CTPA Scans
• Transfer Learning with Edge Attention for Prostate MRI Segmentation
• DoDNet: Learning to segment multi-organ and tumors from multiple partially labeled datasets
• Learning Semantics-enriched Representation via Self-discovery, Self-classification, and Self-restoration
• Towards Foundation Models and Few-Shot Parameter-Efficient Fine-Tuning for Volumetric Organ Segmentation
• Revisiting Rubik's Cube: Self-supervised Learning with Volume-wise Transformation for 3D Medical Image Segmentation
• Automated lung segmentation from CT images of normal and COVID-19 pneumonia patients
• M3Lung-Sys: A Deep Learning System for Multi-Class Lung Pneumonia Screening from CT Imaging
• Generalist Models in Medical Image Segmentation: A Survey and Performance Comparison with Task-Specific Approaches
• Recent Advances in Fibrosis and Scar Segmentation from Cardiac MRI: A State-of-the-Art Review and Future Perspectives
• On Robustness of Lane Detection Models to Physical-World Adversarial Attacks in Autonomous Driving
• Multi-Class 3D Object Detection Within Volumetric 3D Computed Tomography Baggage Security Screening Imagery
• Uncertainty estimations methods for a deep learning model to aid in clinical decision-making -- a clinician's perspective
• Learning Non-Unique Segmentation with Reward-Penalty Dice Loss
• An Accelerated Pipeline for Multi-label Renal Pathology Image Segmentation at the Whole Slide Image Level
• Transferable Visual Words: Exploiting the Semantics of Anatomical Patterns for Self-supervised Learning
• ImageNet Pre-training also Transfers Non-Robustness
• On the Robustness of Pretraining and Self-Supervision for a Deep Learning-based Analysis of Diabetic Retinopathy
• Multi-task Semi-supervised Learning for Pulmonary Lobe Segmentation
• Revisiting 3D Context Modeling with Supervised Pre-training for Universal Lesion Detection in CT Slices
• Pathology-Aware Generative Adversarial Networks for Medical Image Augmentation
• On the Evaluation of Prohibited Item Classification and Detection in Volumetric 3D Computed Tomography Baggage Security Screening Imagery
• DeepPrognosis: Preoperative Prediction of Pancreatic Cancer Survival and Surgical Margin via Contrast-Enhanced CT Imaging
• DeepAtrophy: Teaching a Neural Network to Differentiate Progressive Changes from Noise on Longitudinal MRI in Alzheimer's Disease