Recent advances and clinical applications of deep learning in medical image analysis

Abstract

Deep learning has received extensive research interest in developing new medical image processing algorithms, and deep learning based models have been remarkably successful in a variety of medical imaging tasks to support disease detection and diagnosis. Despite the success, the further improvement of deep learning models in medical image analysis is majorly bottlenecked by the lack of large-sized and well-annotated datasets. In the past five years, many studies have focused on addressing this challenge. In this paper, we reviewed and summarized these recent studies to provide a comprehensive overview of applying deep learning methods in various medical image analysis tasks. Especially, we emphasize the latest progress and contributions of state-of-the-art unsupervised and semi-supervised deep learning in medical image analysis, which are summarized based on different application scenarios, including classification, segmentation, detection, and image registration. We also discuss the major technical challenges and suggest the possible solutions in future research efforts.

arXiv commentTo appear in the journal Medical Image Analysis. The registration section was revised

References in corpus (101)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• A Survey on Deep Learning in Medical Image Analysis
• Neural Machine Translation by Jointly Learning to Align and Translate
• Automated Design of Deep Learning Methods for Biomedical Image Segmentation
• Conditional Generative Adversarial Nets
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• A Simple Framework for Contrastive Learning of Visual Representations
• mixup: Beyond Empirical Risk Minimization
• Attention U-Net: Learning Where to Look for the Pancreas
• Neural Architecture Search with Reinforcement Learning
• R-FCN: Object Detection via Region-based Fully Convolutional Networks
• Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?
• The Future of Digital Health with Federated Learning
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations
• An Introduction to Variational Autoencoders
• FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
• VoxelMorph: A Learning Framework for Deformable Medical Image Registration
• Improved Baselines with Momentum Contrastive Learning
• GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classification
• Generative Adversarial Network in Medical Imaging: A Review
• Neural Architecture Search: A Survey
• Unsupervised Data Augmentation for Consistency Training
• Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
• Semi-Supervised Learning with Deep Generative Models
• Unsupervised Representation Learning by Predicting Image Rotations
• Improved Training of Wasserstein GANs
• Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the LUNA16 challenge
• Focal Loss for Dense Object Detection
• OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks
• Medical Image Analysis using Convolutional Neural Networks: A Review
• Domain-adversarial neural networks to address the appearance variability of histopathology images
• Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans
• Training Generative Adversarial Networks with Limited Data
• Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation
• A Deep Learning Framework for Unsupervised Affine and Deformable Image Registration
• Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks
• Deep Learning in Medical Image Registration: A Review
• SegAN: Adversarial Network with Multi-scale $L_1$ Loss for Medical Image Segmentation
• f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization
• Improving Computer-aided Detection using Convolutional Neural Networks and Random View Aggregation
• Semi-Supervised Learning with Ladder Networks
• NiftyNet: a deep-learning platform for medical imaging
• Associative Embedding: End-to-End Learning for Joint Detection and Grouping
• Deep Learning in Medical Image Registration: A Survey
• Semi-Supervised Learning with Generative Adversarial Networks
• Evaluate the Malignancy of Pulmonary Nodules Using the 3D Deep Leaky Noisy-or Network
• Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders
• Weakly-Supervised Convolutional Neural Networks for Multimodal Image Registration
• Recurrent Residual Convolutional Neural Network based on U-Net (R2U-Net) for Medical Image Segmentation
• Shape-aware Semi-supervised 3D Semantic Segmentation for Medical Images
• MMD GAN: Towards Deeper Understanding of Moment Matching Network
• Confidence Calibration and Predictive Uncertainty Estimation for Deep Medical Image Segmentation
• Med3D: Transfer Learning for 3D Medical Image Analysis
• Describing Multimedia Content using Attention-based Encoder--Decoder Networks
• Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentation
• Models Genesis
• A Survey on Incorporating Domain Knowledge into Deep Learning for Medical Image Analysis
• Unsupervised 3D End-to-End Medical Image Registration with Volume Tweening Network
• Semi-supervised Medical Image Classification with Relation-driven Self-ensembling Model
• Contrastive Learning of Medical Visual Representations from Paired Images and Text
• A Topological Loss Function for Deep-Learning based Image Segmentation using Persistent Homology
• UNet++: A Nested U-Net Architecture for Medical Image Segmentation
• Diagnose like a Radiologist: Attention Guided Convolutional Neural Network for Thorax Disease Classification
• Contrastive learning of global and local features for medical image segmentation with limited annotations
• Triple Generative Adversarial Nets
• UNETR: Transformers for 3D Medical Image Segmentation
• A Theoretical Analysis of Contrastive Unsupervised Representation Learning
• Momentum Contrastive Learning for Few-Shot COVID-19 Diagnosis from Chest CT Images
• CenterNet: Keypoint Triplets for Object Detection
• 3D Self-Supervised Methods for Medical Imaging
• Learning Deep Similarity Metric for 3D MR-TRUS Registration
• The Importance of Skip Connections in Biomedical Image Segmentation
• TransFuse: Fusing Transformers and CNNs for Medical Image Segmentation
• Cascaded Partial Decoder for Fast and Accurate Salient Object Detection
• You Only Learn Once: Universal Anatomical Landmark Detection
• CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentation
• Reverse Attention for Salient Object Detection
• MoCo-CXR: MoCo Pretraining Improves Representation and Transferability of Chest X-ray Models
• Accurate Pulmonary Nodule Detection in Computed Tomography Images Using Deep Convolutional Neural Networks
• Iterative Amortized Inference
• Dual Path Networks
• Propagating Uncertainty in Multi-Stage Bayesian Convolutional Neural Networks with Application to Pulmonary Nodule Detection
• Self-Supervised Learning for Cardiac MR Image Segmentation by Anatomical Position Prediction
• Semi-Supervised Deep Learning for Abnormality Classification in Retinal Images
• MedAug: Contrastive learning leveraging patient metadata improves representations for chest X-ray interpretation
• Transferring GANs: generating images from limited data
• Cross-view Relation Networks for Mammogram Mass Detection
• Self-supervised Pre-training with Hard Examples Improves Visual Representations
• Conditional Infilling GANs for Data Augmentation in Mammogram Classification
• Self-Supervised Learning for Spinal MRIs
• Self-supervised Feature Learning for 3D Medical Images by Playing a Rubik's Cube
• Unsupervised Deformable Image Registration Using Cycle-Consistent CNN
• Joint Learning for Pulmonary Nodule Segmentation, Attributes and Malignancy Prediction
• Revisiting Rubik's Cube: Self-supervised Learning with Volume-wise Transformation for 3D Medical Image Segmentation
• PAFNet: An Efficient Anchor-Free Object Detector Guidance
• Improving Deep Lesion Detection Using 3D Contextual and Spatial Attention
• ULDor: A Universal Lesion Detector for CT Scans with Pseudo Masks and Hard Negative Example Mining
• Lymph Node Gross Tumor Volume Detection and Segmentation via Distance-based Gating using 3D CT/PET Imaging in Radiotherapy
• Renal Cell Carcinoma Detection and Subtyping with Minimal Point-Based Annotation in Whole-Slide Images

Cited by in corpus (22)

• CLIP-Driven Universal Model for Organ Segmentation and Tumor Detection
• Self-supervised learning methods and applications in medical imaging analysis: A survey
• A Foundation Language-Image Model of the Retina (FLAIR): Encoding Expert Knowledge in Text Supervision
• Is attention all you need in medical image analysis? A review
• Virtual histological staining of unlabeled autopsy tissue
• Can autism be diagnosed with AI?
• Deep Learning for Pancreas Segmentation: a Systematic Review
• A Deep Registration Method for Accurate Quantification of Joint Space Narrowing Progression in Rheumatoid Arthritis
• Few-shot learning for COVID-19 Chest X-Ray Classification with Imbalanced Data: An Inter vs. Intra Domain Study
• Towards Foundation Models and Few-Shot Parameter-Efficient Fine-Tuning for Volumetric Organ Segmentation
• Gravity Network for end-to-end small lesion detection
• Are Vision xLSTM Embedded UNet More Reliable in Medical 3D Image Segmentation?
• Robust and Explainable Framework to Address Data Scarcity in Diagnostic Imaging
• Unified Multi-modal Diagnostic Framework with Reconstruction Pre-training and Heterogeneity-combat Tuning
• MedVKAN: Efficient Feature Extraction with Mamba and KAN for Medical Image Segmentation
• Towards the use of multiple ROIs for radiomics-based survival modelling: finding a strategy of aggregating lesions
• Self-Supervised Representation Learning for Nerve Fiber Distribution Patterns in 3D-PLI
• Adaptive Input-image Normalization for Solving the Mode Collapse Problem in GAN-based X-ray Images
• VIS-MAE: An Efficient Self-supervised Learning Approach on Medical Image Segmentation and Classification
• Label-free evaluation of lung and heart transplant biopsies using tissue autofluorescence-based virtual staining
• AGNES: Abstraction-guided Framework for Deep Neural Networks Security
• Trustworthy Data-driven Chronological Age Estimation from Panoramic Dental Images