Multi-SpacePhish: Extending the Evasion-space of Adversarial Attacks against Phishing Website Detectors using Machine Learning

Abstract

Existing literature on adversarial Machine Learning (ML) focuses either on showing attacks that break every ML model, or defenses that withstand most attacks. Unfortunately, little consideration is given to the actual feasibility of the attack or the defense. Moreover, adversarial samples are often crafted in the "feature-space", making the corresponding evaluations of questionable value. Simply put, the current situation does not allow to estimate the actual threat posed by adversarial attacks, leading to a lack of secure ML systems. We aim to clarify such confusion in this paper. By considering the application of ML for Phishing Website Detection (PWD), we formalize the "evasion-space" in which an adversarial perturbation can be introduced to fool a ML-PWD -- demonstrating that even perturbations in the "feature-space" are useful. Then, we propose a realistic threat model describing evasion attacks against ML-PWD that are cheap to stage, and hence intrinsically more attractive for real phishers. After that, we perform the first statistically validated assessment of state-of-the-art ML-PWD against 12 evasion attacks. Our evaluation shows (i) the true efficacy of evasion attempts that are more likely to occur; and (ii) the impact of perturbations crafted in different evasion-spaces. Our realistic evasion attempts induce a statistically significant degradation (3-10% at p<0.05), and their cheap cost makes them a subtle threat. Notably, however, some ML-PWD are immune to our most realistic attacks (p=0.22). Finally, as an additional contribution of this journal publication, we are the first to consider the intriguing case wherein an attacker introduces perturbations in multiple evasion-spaces at the same time. These new results show that simultaneously applying perturbations in the problem- and feature-space can cause a drop in the detection rate from 0.95 to 0.

References in corpus (16)

• One pixel attack for fooling deep neural networks
• Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning
• On Evaluating Adversarial Robustness
• The Role of Machine Learning in Cybersecurity
• Defensive Distillation is Not Robust to Adversarial Examples
• Modeling Realistic Adversarial Attacks against Network Intrusion Detection Systems
• Towards Benchmark Datasets for Machine Learning Based Website Phishing Detection: An experimental study
• Malware Detection using Machine Learning and Deep Learning
• Black-box Adversarial Attacks on Commercial Speech Platforms with Minimal Information
• Gotta Catch 'Em All: Using Honeypots to Catch Adversarial Attacks on Neural Networks
• AdVersarial: Perceptual Ad Blocking meets Adversarial Machine Learning
• Multi-SpacePhish: Extending the Evasion-space of Adversarial Attacks against Phishing Website Detectors using Machine Learning
• SoK: The Impact of Unlabelled Data in Cyberthreat Detection
• Poisoning the Unlabeled Dataset of Semi-Supervised Learning
• Almost Tight L0-norm Certified Robustness of Top-k Predictions against Adversarial Perturbations
• Exploiting the Inherent Limitation of L0 Adversarial Examples

Cited by in corpus (4)

• Multi-SpacePhish: Extending the Evasion-space of Adversarial Attacks against Phishing Website Detectors using Machine Learning
• The Impact of Emerging Phishing Threats: Assessing Quishing and LLM-generated Phishing Emails against Organizations
• Raze to the Ground: Query-Efficient Adversarial HTML Attacks on Machine-Learning Phishing Webpage Detectors
• "Are Adversarial Phishing Webpages a Threat in Reality?" Understanding the Users' Perception of Adversarial Webpages