A Survey on Active Learning and Human-in-the-Loop Deep Learning for Medical Image Analysis

Abstract

Fully automatic deep learning has become the state-of-the-art technique for many tasks including image acquisition, analysis and interpretation, and for the extraction of clinically useful information for computer-aided detection, diagnosis, treatment planning, intervention and therapy. However, the unique challenges posed by medical image analysis suggest that retaining a human end user in any deep learning enabled system will be beneficial. In this review we investigate the role that humans might play in the development and deployment of deep learning enabled diagnostic applications and focus on techniques that will retain a significant input from a human end user. Human-in-the-Loop computing is an area that we see as increasingly important in future research due to the safety-critical nature of working in the medical domain. We evaluate four key areas that we consider vital for deep learning in the clinical practice: (1) Active Learning to choose the best data to annotate for optimal model performance; (2) Interaction with model outputs - using iterative feedback to steer models to optima for a given prediction and offering meaningful ways to interpret and respond to predictions; (3) Practical considerations - developing full scale applications and the key considerations that need to be made before deployment; (4) Future Prospective and Unanswered Questions - knowledge gaps and related research fields that will benefit human-in-the-loop computing as they evolve. We offer our opinions on the most promising directions of research and how various aspects of each area might be unified towards common goals.

Medical Image Analysis Volume 71 2021 https://doi.org/10.1016/j.media.2021.102062

References in corpus (12)

• Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?
• Showing Academic Performance Predictions during Term Planning: Effects on Students' Decisions, Behaviors, and Preferences
• Cost-Effective Active Learning for Deep Image Classification
• Deep Bayesian Active Learning with Image Data
• A scoping review of transfer learning research on medical image analysis using ImageNet
• Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentation
• Towards the Augmented Pathologist: Challenges of Explainable-AI in Digital Pathology
• Active One-shot Learning
• Learning Algorithms for Active Learning
• On Statistical Bias In Active Learning: How and When To Fix It
• Annotation-cost Minimization for Medical Image Segmentation using Suggestive Mixed Supervision Fully Convolutional Networks
• Employing Weak Annotations for Medical Image Analysis Problems

Cited by in corpus (35)

• A Survey of Human-in-the-loop for Machine Learning
• Challenges in Deploying Machine Learning: a Survey of Case Studies
• Diminishing Uncertainty within the Training Pool: Active Learning for Medical Image Segmentation
• General Purpose Artificial Intelligence Systems (GPAIS): Properties, Definition, Taxonomy, Societal Implications and Responsible Governance
• Expert enhanced dynamic time warping based anomaly detection
• Medical Image Classification Using Transfer Learning and Chaos Game Optimization on the Internet of Medical Things
• Towards contrast-agnostic soft segmentation of the spinal cord
• Improving deep learning with prior knowledge and cognitive models: A survey on enhancing explainability, adversarial robustness and zero-shot learning
• Deep Learning Body Region Classification of MRI and CT examinations
• Addressing UX Practitioners' Challenges in Designing ML Applications: an Interactive Machine Learning Approach
• Interpolation-Split: a data-centric deep learning approach with big interpolated data to boost airway segmentation performance
• Automatic Segmentation of the Spinal Cord Nerve Rootlets
• In the Picture: Medical Imaging Datasets, Artifacts, and their Living Review
• Gone Fishing: Neural Active Learning with Fisher Embeddings
• Active Learning for Identifying Disaster-Related Tweets: A Comparison with Keyword Filtering and Generic Fine-Tuning
• Generating 3D Bio-Printable Patches Using Wound Segmentation and Reconstruction to Treat Diabetic Foot Ulcers
• Advances in Medical Image Segmentation: A Comprehensive Survey with a Focus on Lumbar Spine Applications
• Reducing Annotating Load: Active Learning with Synthetic Images in Surgical Instrument Segmentation
• Autonomous Small-Angle Scattering for Accelerated Soft Material Formulation Optimization
• Categorising the World into Local Climate Zones -- Towards Quantifying Labelling Uncertainty for Machine Learning Models
• Complex QA and language models hybrid architectures, Survey
• Entity Matching by Pool-based Active Learning
• Active learning for object detection in high-resolution satellite images
• Adaptive Multi-scale Online Likelihood Network for AI-assisted Interactive Segmentation
• Sustaining model performance for covid-19 detection from dynamic audio data: Development and evaluation of a comprehensive drift-adaptive framework
• Fast and Scalable Image Search For Histology
• Temporal Uncertainty Localization to Enable Human-in-the-loop Analysis of Dynamic Contrast-enhanced Cardiac MRI Datasets
• Classifying Breast Histopathology Images with a Ductal Instance-Oriented Pipeline
• A Tale Of Two Long Tails
• Improving traffic sign recognition by active search
• Towards Using Active Learning Methods for Human-Seat Interactions To Generate Realistic Occupant Motion
• Continual Active Learning for Efficient Adaptation of Machine Learning Models to Changing Image Acquisition
• Augmenting Chest X-ray Datasets with Non-Expert Annotations
• Comprehensive framework for evaluation of deep neural networks in detection and quantification of lymphoma from PET/CT images: clinical insights, pitfalls, and observer agreement analyses
• Active learning for medical code assignment