Adversarial Attacks Against Medical Deep Learning Systems

Abstract

The discovery of adversarial examples has raised concerns about the practical deployment of deep learning systems. In this paper, we demonstrate that adversarial examples are capable of manipulating deep learning systems across three clinical domains. For each of our representative medical deep learning classifiers, both white and black box attacks were highly successful. Our models are representative of the current state of the art in medical computer vision and, in some cases, directly reflect architectures already seeing deployment in real world clinical settings. In addition to the technical contribution of our paper, we synthesize a large body of knowledge about the healthcare system to argue that medicine may be uniquely susceptible to adversarial attacks, both in terms of monetary incentives and technical vulnerability. To this end, we outline the healthcare economy and the incentives it creates for fraud and provide concrete examples of how and why such attacks could be realistically carried out. We urge practitioners to be aware of current vulnerabilities when deploying deep learning systems in clinical settings, and encourage the machine learning community to further investigate the domain-specific characteristics of medical learning systems.

References in corpus (18)

• mixup: Beyond Empirical Risk Minimization
• Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks
• Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning
• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
• CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Synthesizing Robust Adversarial Examples
• Certified Defenses against Adversarial Examples
• Robust Physical-World Attacks on Deep Learning Models
• The Space of Transferable Adversarial Examples
• Technical Report on the CleverHans v2.1.0 Adversarial Examples Library
• Robustness May Be at Odds with Accuracy
• Adversarial Examples for Evaluating Reading Comprehension Systems
• NO Need to Worry about Adversarial Examples in Object Detection in Autonomous Vehicles
• A Dual Approach to Scalable Verification of Deep Networks
• Detecting hip fractures with radiologist-level performance using deep neural networks
• Generalizability vs. Robustness: Adversarial Examples for Medical Imaging
• Assessing Threat of Adversarial Examples on Deep Neural Networks

Cited by in corpus (37)

• Deep Learning for Anomaly Detection: A Survey
• A Review of Challenges and Opportunities in Machine Learning for Health
• Adversarial Examples: Opportunities and Challenges
• Adversarial Example Detection for DNN Models: A Review and Experimental Comparison
• Adversarial Attack Vulnerability of Medical Image Analysis Systems: Unexplored Factors
• Security and Privacy Issues in Deep Learning
• Towards Evaluating the Robustness of Deep Diagnostic Models by Adversarial Attack
• A Survey of Safety and Trustworthiness of Deep Neural Networks: Verification, Testing, Adversarial Attack and Defence, and Interpretability
• Adversarial Examples in Modern Machine Learning: A Review
• Adversarial Attacks and Defenses in Physiological Computing: A Systematic Review
• Measuring the Transferability of Adversarial Examples
• Anomalous Example Detection in Deep Learning: A Survey
• Interpreting Deep Learning: The Machine Learning Rorschach Test?
• Now You See It, Now You Dont: Adversarial Vulnerabilities in Computational Pathology
• Bias Field Poses a Threat to DNN-based X-Ray Recognition
• Developing Future Human-Centered Smart Cities: Critical Analysis of Smart City Security, Interpretability, and Ethical Challenges
• Exploring Adversarial Examples: Patterns of One-Pixel Attacks
• DAPAS : Denoising Autoencoder to Prevent Adversarial attack in Semantic Segmentation
• Defending against adversarial attacks on medical imaging AI system, classification or detection?
• Explainability Matters: Backdoor Attacks on Medical Imaging
• Vulnerability Analysis of Chest X-Ray Image Classification Against Adversarial Attacks
• Adversarial Examples for Electrocardiograms
• Adversarial Transfer Attacks With Unknown Data and Class Overlap
• MeDaS: An open-source platform as service to help break the walls between medicine and informatics
• Medical Aegis: Robust adversarial protectors for medical images
• Adversarial Robustness Study of Convolutional Neural Network for Lumbar Disk Shape Reconstruction from MR images
• Adversarial Heart Attack: Neural Networks Fooled to Segment Heart Symbols in Chest X-Ray Images
• Techniques for Adversarial Examples Threatening the Safety of Artificial Intelligence Based Systems
• Multi-stage Optimization based Adversarial Training
• A Hierarchical Feature Constraint to Camouflage Medical Adversarial Attacks
• Improving learnability of neural networks: adding supplementary axes to disentangle data representation
• Non-Determinism in Neural Networks for Adversarial Robustness
• Application of Adversarial Examples to Physical ECG Signals
• Deep Bayesian Image Set Classification: A Defence Approach against Adversarial Attacks
• The Effect of Prior Lipschitz Continuity on the Adversarial Robustness of Bayesian Neural Networks
• Intelligent image synthesis to attack a segmentation CNN using adversarial learning
• Attack-agnostic Adversarial Detection on Medical Data Using Explainable Machine Learning