Responsible and Regulatory Conform Machine Learning for Medicine: A Survey of Challenges and Solutions

Abstract

Machine learning is expected to fuel significant improvements in medical care. To ensure that fundamental principles such as beneficence, respect for human autonomy, prevention of harm, justice, privacy, and transparency are respected, medical machine learning systems must be developed responsibly. Many high-level declarations of ethical principles have been put forth for this purpose, but there is a severe lack of technical guidelines explicating the practical consequences for medical machine learning. Similarly, there is currently considerable uncertainty regarding the exact regulatory requirements placed upon medical machine learning systems. This survey provides an overview of the technical and procedural challenges involved in creating medical machine learning systems responsibly and in conformity with existing regulations, as well as possible solutions to address these challenges. First, a brief review of existing regulations affecting medical machine learning is provided, showing that properties such as safety, robustness, reliability, privacy, security, transparency, explainability, and nondiscrimination are all demanded already by existing law and regulations - albeit, in many cases, to an uncertain degree. Next, the key technical obstacles to achieving these desirable properties are discussed, as well as important techniques to overcome these obstacles in the medical context. We notice that distribution shift, spurious correlations, model underspecification, uncertainty quantification, and data scarcity represent severe challenges in the medical context. Promising solution approaches include the use of large and representative datasets and federated learning as a means to that end, the careful exploitation of domain knowledge, the use of inherently transparent models, comprehensive out-of-distribution model testing and verification, as well as algorithmic impact assessments.

References in corpus (61)

• SMOTE: Synthetic Minority Over-sampling Technique
• Deep Learning with Differential Privacy
• Artificial Intelligence: the global landscape of ethics guidelines
• Federated Learning: Challenges, Methods, and Future Directions
• Towards A Rigorous Science of Interpretable Machine Learning
• The Future of Digital Health with Federated Learning
• A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI
• European Union regulations on algorithmic decision-making and a "right to explanation"
• Shortcut Learning in Deep Neural Networks
• Model Cards for Model Reporting
• An overview of deep learning in medical imaging focusing on MRI
• Principles alone cannot guarantee ethical AI
• Confounding variables can degrade generalization performance of radiological deep learning models
• Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications
• Unmasking Clever Hans Predictors and Assessing What Machines Really Learn
• A Survey on Distributed Machine Learning
• Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model
• Improving fairness in machine learning systems: What do industry practitioners need?
• Questioning the AI: Informing Design Practices for Explainable AI User Experiences
• What do we need to build explainable AI systems for the medical domain?
• A review of domain adaptation without target labels
• On Evaluating Adversarial Robustness
• Differentially Private Empirical Risk Minimization
• The What-If Tool: Interactive Probing of Machine Learning Models
• Underspecification Presents Challenges for Credibility in Modern Machine Learning
• Towards Robust Interpretability with Self-Explaining Neural Networks
• Deep Learning is Robust to Massive Label Noise
• Technical Report on the CleverHans v2.1.0 Adversarial Examples Library
• Can You Really Backdoor Federated Learning?
• Causality matters in medical imaging
• InterpretML: A Unified Framework for Machine Learning Interpretability
• Where Responsible AI meets Reality: Practitioner Perspectives on Enablers for shifting Organizational Practices
• Concept Whitening for Interpretable Image Recognition
• One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques
• A Review of Challenges and Opportunities in Machine Learning for Health
• Explanation in Human-AI Systems: A Literature Meta-Review, Synopsis of Key Ideas and Publications, and Bibliography for Explainable AI
• Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims
• Regularization for Deep Learning: A Taxonomy
• On the Robustness of the CVPR 2018 White-Box Adversarial Example Defenses
• What is the Effect of Importance Weighting in Deep Learning?
• Fairness for Unobserved Characteristics: Insights from Technological Impacts on Queer Communities
• Split Learning for collaborative deep learning in healthcare
• Simulation assisted machine learning
• Towards Practical Verification of Machine Learning: The Case of Computer Vision Systems
• Interpretability and Explainability: A Machine Learning Zoo Mini-tour
• Contextual Explanation Networks
• Principles to Practices for Responsible AI: Closing the Gap
• Monotonic Calibrated Interpolated Look-Up Tables
• Why Fairness Cannot Be Automated: Bridging the Gap Between EU Non-Discrimination Law and AI
• Causal Learning and Explanation of Deep Neural Networks via Autoencoded Activations
• Survey on Causal-based Machine Learning Fairness Notions
• Formal Verification of Input-Output Mappings of Tree Ensembles
• An Evaluation Toolkit to Guide Model Selection and Cohort Definition in Causal Inference
• Formal Verification of Decision-Tree Ensemble Model and Detection of its Violating-input-value Ranges
• Calibrating Healthcare AI: Towards Reliable and Interpretable Deep Predictive Models
• Assessing differentially private deep learning with Membership Inference
• Impact of Accuracy on Model Interpretations
• Algorithmic Bias and Data Bias: Understanding the Relation between Distributionally Robust Optimization and Data Curation
• Contestable Black Boxes
• Calibrating Deep Neural Network Classifiers on Out-of-Distribution Datasets
• Safety design concepts for statistical machine learning components toward accordance with functional safety standards

Cited by in corpus (3)

• Multimodal Healthcare AI: Identifying and Designing Clinically Relevant Vision-Language Applications for Radiology
• Twenty-Four Years of Empirical Research on Trust in AI: A Bibliometric Review of Trends, Overlooked Issues, and Future Directions
• Robust and Explainable Framework to Address Data Scarcity in Diagnostic Imaging