Adversarial Attack Vulnerability of Medical Image Analysis Systems: Unexplored Factors

Abstract

Adversarial attacks are considered a potentially serious security threat for machine learning systems. Medical image analysis (MedIA) systems have recently been argued to be vulnerable to adversarial attacks due to strong financial incentives and the associated technological infrastructure. In this paper, we study previously unexplored factors affecting adversarial attack vulnerability of deep learning MedIA systems in three medical domains: ophthalmology, radiology, and pathology. We focus on adversarial black-box settings, in which the attacker does not have full access to the target model and usually uses another model, commonly referred to as surrogate model, to craft adversarial examples. We consider this to be the most realistic scenario for MedIA systems. Firstly, we study the effect of weight initialization (ImageNet vs. random) on the transferability of adversarial attacks from the surrogate model to the target model. Secondly, we study the influence of differences in development data between target and surrogate models. We further study the interaction of weight initialization and data differences with differences in model architecture. All experiments were done with a perturbation degree tuned to ensure maximal transferability at minimal visual perceptibility of the attacks. Our experiments show that pre-training may dramatically increase the transferability of adversarial examples, even when the target and surrogate's architectures are different: the larger the performance gain using pre-training, the larger the transferability. Differences in the development data between target and surrogate models considerably decrease the performance of the attack; this decrease is further amplified by difference in the model architecture. We believe these factors should be considered when developing security-critical MedIA systems planned to be deployed in clinical practice.

arXiv commentFirst three authors contributed equally

References in corpus (33)

• A Survey on Deep Learning in Medical Image Analysis
• Explaining and Harnessing Adversarial Examples
• Intriguing properties of neural networks
• Ensemble Adversarial Training: Attacks and Defenses
• One pixel attack for fooling deep neural networks
• Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning
• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
• CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Theoretically Principled Trade-off between Robustness and Accuracy
• Delving into Transferable Adversarial Examples and Black-box Attacks
• Automated Gleason Grading of Prostate Biopsies using Deep Learning
• On Evaluating Adversarial Robustness
• Understanding Adversarial Attacks on Deep Learning Based Medical Image Analysis Systems
• The Space of Transferable Adversarial Examples
• Using Pre-Training Can Improve Model Robustness and Uncertainty
• Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
• Robustness May Be at Odds with Accuracy
• PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examples
• Skip Connections Matter: On the Transferability of Adversarial Examples Generated with ResNets
• Understanding Measures of Uncertainty for Adversarial Example Detection
• Adversarial Patch
• Computer aided detection of tuberculosis on chest radiographs: An evaluation of the CAD4TB v6 system
• Adversarial Attacks Against Medical Deep Learning Systems
• Evaluation of a deep learning system for the joint automated detection of diabetic retinopathy and age-related macular degeneration
• Dropout Inference in Bayesian Neural Networks with Alpha-divergences
• Automatic brain tumor grading from MRI data using convolutional neural networks and quality assessment
• Pseudo-healthy synthesis with pathology disentanglement and adversarial learning
• Attacks Meet Interpretability: Attribute-steered Detection of Adversarial Samples
• Deep learning assessment of breast terminal duct lobular unit involution: towards automated prediction of breast cancer risk
• Generalizability vs. Robustness: Adversarial Examples for Medical Imaging
• Is AmI (Attacks Meet Interpretability) Robust to Adversarial Examples?
• Vulnerability Analysis of Chest X-Ray Image Classification Against Adversarial Attacks

Cited by in corpus (10)

• MedViT: A Robust Vision Transformer for Generalized Medical Image Classification
• How Deep Learning Sees the World: A Survey on Adversarial Attacks & Defenses
• Data synthesis and adversarial networks: A review and meta-analysis in cancer imaging
• Survey on Adversarial Attack and Defense for Medical Image Analysis: Methods and Challenges
• When and How to Fool Explainable Models (and Humans) with Adversarial Examples
• Adversarial Attack Driven Data Augmentation for Accurate And Robust Medical Image Segmentation
• Improving Robustness and Reliability in Medical Image Classification with Latent-Guided Diffusion and Nested-Ensembles
• CoRPA: Adversarial Image Generation for Chest X-rays Using Concept Vector Perturbations and Generative Models
• Revisiting Hidden Representations in Transfer Learning for Medical Imaging
• Universal and Transferable Attacks on Pathology Foundation Models