A Survey on Active Learning and Human-in-the-Loop Deep Learning for Medical Image Analysis

Abstract

Fully automatic deep learning has become the state-of-the-art technique for many tasks including image acquisition, analysis and interpretation, and for the extraction of clinically useful information for computer-aided detection, diagnosis, treatment planning, intervention and therapy. However, the unique challenges posed by medical image analysis suggest that retaining a human end user in any deep learning enabled system will be beneficial. In this review we investigate the role that humans might play in the development and deployment of deep learning enabled diagnostic applications and focus on techniques that will retain a significant input from a human end user. Human-in-the-Loop computing is an area that we see as increasingly important in future research due to the safety-critical nature of working in the medical domain. We evaluate four key areas that we consider vital for deep learning in the clinical practice: (1) Active Learning to choose the best data to annotate for optimal model performance; (2) Interaction with model outputs - using iterative feedback to steer models to optima for a given prediction and offering meaningful ways to interpret and respond to predictions; (3) Practical considerations - developing full scale applications and the key considerations that need to be made before deployment; (4) Future Prospective and Unanswered Questions - knowledge gaps and related research fields that will benefit human-in-the-loop computing as they evolve. We offer our opinions on the most promising directions of research and how various aspects of each area might be unified towards common goals.

Medical Image Analysis Volume 71 2021 https://doi.org/10.1016/j.media.2021.102062

References in corpus (35)

• A Survey on Deep Learning in Medical Image Analysis
• Attention U-Net: Learning Where to Look for the Pancreas
• Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
• Conditional Random Fields as Recurrent Neural Networks
• European Union regulations on algorithmic decision-making and a "right to explanation"
• An overview of deep learning in medical imaging focusing on MRI
• Generative Adversarial Network in Medical Imaging: A Review
• Showing Academic Performance Predictions during Term Planning: Effects on Students' Decisions, Behaviors, and Preferences
• Interactive Medical Image Segmentation using Deep Learning with Image-specific Fine-tuning
• Cost-Effective Active Learning for Deep Image Classification
• Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks
• Transfusion: Understanding Transfer Learning for Medical Imaging
• Deep Bayesian Active Learning with Image Data
• A scoping review of transfer learning research on medical image analysis using ImageNet
• DeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation
• Deep Learning for Multi-Task Medical Image Segmentation in Multiple Modalities
• Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentation
• BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning
• Learning Active Learning from Data
• Automatic brain tumor grading from MRI data using convolutional neural networks and quality assessment
• Towards the Augmented Pathologist: Challenges of Explainable-AI in Digital Pathology
• Active One-shot Learning
• Active, Continual Fine Tuning of Convolutional Neural Networks for Reducing Annotation Efforts
• Active Learning from Weak and Strong Labelers
• Learning Algorithms for Active Learning
• Towards continual learning in medical imaging
• Learning under Distributed Weak Supervision
• On Statistical Bias In Active Learning: How and When To Fix It
• Active Learning for Segmentation by Optimizing Content Information for Maximal Entropy
• Geometry in Active Learning for Binary and Multi-class Image Segmentation
• MedAL: Deep Active Learning Sampling Method for Medical Image Analysis
• Annotation-cost Minimization for Medical Image Segmentation using Suggestive Mixed Supervision Fully Convolutional Networks
• Employing Weak Annotations for Medical Image Analysis Problems
• Gaze2Segment: A Pilot Study for Integrating Eye-Tracking Technology into Medical Image Segmentation

Cited by in corpus (36)

• A Survey of Human-in-the-loop for Machine Learning
• Challenges in Deploying Machine Learning: a Survey of Case Studies
• Diminishing Uncertainty within the Training Pool: Active Learning for Medical Image Segmentation
• General Purpose Artificial Intelligence Systems (GPAIS): Properties, Definition, Taxonomy, Societal Implications and Responsible Governance
• Active, Continual Fine Tuning of Convolutional Neural Networks for Reducing Annotation Efforts
• Expert enhanced dynamic time warping based anomaly detection
• Medical Image Classification Using Transfer Learning and Chaos Game Optimization on the Internet of Medical Things
• Towards contrast-agnostic soft segmentation of the spinal cord
• Improving deep learning with prior knowledge and cognitive models: A survey on enhancing explainability, adversarial robustness and zero-shot learning
• Deep Learning Body Region Classification of MRI and CT examinations
• Addressing UX Practitioners' Challenges in Designing ML Applications: an Interactive Machine Learning Approach
• Interpolation-Split: a data-centric deep learning approach with big interpolated data to boost airway segmentation performance
• Gone Fishing: Neural Active Learning with Fisher Embeddings
• In the Picture: Medical Imaging Datasets, Artifacts, and their Living Review
• Automatic Segmentation of the Spinal Cord Nerve Rootlets
• Active Learning for Identifying Disaster-Related Tweets: A Comparison with Keyword Filtering and Generic Fine-Tuning
• Generating 3D Bio-Printable Patches Using Wound Segmentation and Reconstruction to Treat Diabetic Foot Ulcers
• Reducing Annotating Load: Active Learning with Synthetic Images in Surgical Instrument Segmentation
• Advances in Medical Image Segmentation: A Comprehensive Survey with a Focus on Lumbar Spine Applications
• Autonomous Small-Angle Scattering for Accelerated Soft Material Formulation Optimization
• Categorising the World into Local Climate Zones -- Towards Quantifying Labelling Uncertainty for Machine Learning Models
• Complex QA and language models hybrid architectures, Survey
• Active learning for object detection in high-resolution satellite images
• Entity Matching by Pool-based Active Learning
• Adaptive Multi-scale Online Likelihood Network for AI-assisted Interactive Segmentation
• Sustaining model performance for covid-19 detection from dynamic audio data: Development and evaluation of a comprehensive drift-adaptive framework
• Fast and Scalable Image Search For Histology
• Temporal Uncertainty Localization to Enable Human-in-the-loop Analysis of Dynamic Contrast-enhanced Cardiac MRI Datasets
• Classifying Breast Histopathology Images with a Ductal Instance-Oriented Pipeline
• Towards Using Active Learning Methods for Human-Seat Interactions To Generate Realistic Occupant Motion
• Improving traffic sign recognition by active search
• A Tale Of Two Long Tails
• Comprehensive framework for evaluation of deep neural networks in detection and quantification of lymphoma from PET/CT images: clinical insights, pitfalls, and observer agreement analyses
• Active learning for medical code assignment
• Continual Active Learning for Efficient Adaptation of Machine Learning Models to Changing Image Acquisition
• Augmenting Chest X-ray Datasets with Non-Expert Annotations