A Survey on Incorporating Domain Knowledge into Deep Learning for Medical Image Analysis

Abstract

Although deep learning models like CNNs have achieved great success in medical image analysis, the small size of medical datasets remains a major bottleneck in this area. To address this problem, researchers have started looking for external information beyond current available medical datasets. Traditional approaches generally leverage the information from natural images via transfer learning. More recent works utilize the domain knowledge from medical doctors, to create networks that resemble how medical doctors are trained, mimic their diagnostic patterns, or focus on the features or areas they pay particular attention to. In this survey, we summarize the current progress on integrating medical domain knowledge into deep learning models for various tasks, such as disease diagnosis, lesion, organ and abnormality detection, lesion and organ segmentation. For each task, we systematically categorize different kinds of medical domain knowledge that have been utilized and their corresponding integrating methods. We also provide current challenges and directions for future research.

arXiv comment27 pages, 18 figures

References in corpus (47)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• A Survey on Deep Learning in Medical Image Analysis
• Conditional Generative Adversarial Nets
• Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation
• ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases
• How transferable are features in deep neural networks?
• Brain Tumor Segmentation with Deep Neural Networks
• Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?
• Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the LUNA16 challenge
• Unsupervised Domain Adaptation with Residual Transfer Networks
• Domain-adversarial neural networks to address the appearance variability of histopathology images
• Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)
• Anatomically Constrained Neural Networks (ACNN): Application to Cardiac Image Enhancement and Segmentation
• Automatic Liver and Lesion Segmentation in CT Using Cascaded Fully Convolutional Neural Networks and 3D Conditional Random Fields
• Recurrent Residual Convolutional Neural Network based on U-Net (R2U-Net) for Medical Image Segmentation
• On the Automatic Generation of Medical Imaging Reports
• Med3D: Transfer Learning for 3D Medical Image Analysis
• Deep Learning for Multi-Task Medical Image Segmentation in Multiple Modalities
• A Fully Convolutional Neural Network for Cardiac Segmentation in Short-Axis MRI
• Semi-supervised Medical Image Classification with Relation-driven Self-ensembling Model
• MURA: Large Dataset for Abnormality Detection in Musculoskeletal Radiographs
• Diagnose like a Radiologist: Attention Guided Convolutional Neural Network for Thorax Disease Classification
• Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generation
• A Survey on Neural Architecture Search
• Collaborative Unsupervised Domain Adaptation for Medical Image Diagnosis
• Automatic Liver and Tumor Segmentation of CT and MRI Volumes using Cascaded Fully Convolutional Neural Networks
• Deep convolutional networks for pancreas segmentation in CT imaging
• Risk Stratification of Lung Nodules Using 3D CNN-Based Multi-task Learning
• Joint Weakly and Semi-Supervised Deep Learning for Localization and Classification of Masses in Breast Ultrasound Images
• Clinically Accurate Chest X-Ray Report Generation
• Improving Deep Pancreas Segmentation in CT and MRI Images via Recurrent Neural Contextual Learning and Direct Loss Function
• Cardiac Segmentation with Strong Anatomical Guarantees
• Automated sub-cortical brain structure segmentation combining spatial and deep convolutional features
• FocusNetv2: Imbalanced Large and Small Organ Segmentation with Adversarial Shape Constraint for Head and Neck CT Images
• Curriculum semi-supervised segmentation
• Cardiac MRI Segmentation with Strong Anatomical Guarantees
• Learning Shape Priors for Robust Cardiac MR Segmentation from Multi-view Images
• Weakly Supervised Vessel Segmentation in X-ray Angiograms by Self-Paced Learning from Noisy Labels with Suggestive Annotation
• Medical-based Deep Curriculum Learning for Improved Fracture Classification
• Producing radiologist-quality reports for interpretable artificial intelligence
• Combining Shape Priors with Conditional Adversarial Networks for Improved Scapula Segmentation in MR images
• Semi-Supervised Self-Taught Deep Learning for Finger Bones Segmentation
• Organ at Risk Segmentation for Head and Neck Cancer using Stratified Learning and Neural Architecture Search
• Synergistic Reconstruction and Synthesis via Generative Adversarial Networks for Accelerated Multi-Contrast MRI
• Learn like a Pathologist: Curriculum Learning by Annotator Agreement for Histopathology Image Classification
• Detection and Attention: Diagnosing Pulmonary Lung Cancer from CT by Imitating Physicians
• PK-GCN: Prior Knowledge Assisted Image Classification using Graph Convolution Networks

Cited by in corpus (12)

• Recent advances and clinical applications of deep learning in medical image analysis
• Graph-Based Deep Learning for Medical Diagnosis and Analysis: Past, Present and Future
• A Survey on Graph-Based Deep Learning for Computational Histopathology
• A Foundation Language-Image Model of the Retina (FLAIR): Encoding Expert Knowledge in Text Supervision
• CLIP in Medical Imaging: A Survey
• Applications and Techniques for Fast Machine Learning in Science
• The Expert Knowledge combined with AI outperforms AI Alone in Seizure Onset Zone Localization using resting state fMRI
• MuVAM: A Multi-View Attention-based Model for Medical Visual Question Answering
• Unified Multi-modal Diagnostic Framework with Reconstruction Pre-training and Heterogeneity-combat Tuning
• BreakNet: Discontinuity-Resilient Multi-Scale Transformer Segmentation of Retinal Layers
• Multi-task Learning Approach for Intracranial Hemorrhage Prognosis
• CLOG-CD: Curriculum Learning based on Oscillating Granularity of Class Decomposed Medical Image Classification