Security Analysis of Online Centroid Anomaly Detection

Abstract

Security issues are crucial in a number of machine learning applications, especially in scenarios dealing with human activity rather than natural phenomena (e.g., information ranking, spam detection, malware detection, etc.). It is to be expected in such cases that learning algorithms will have to deal with manipulated data aimed at hampering decision making. Although some previous work addressed the handling of malicious data in the context of supervised learning, very little is known about the behavior of anomaly detection methods in such scenarios. In this contribution we analyze the performance of a particular method -- online centroid anomaly detection -- in the presence of adversarial noise. Our analysis addresses the following security-related issues: formalization of learning and attack processes, derivation of an optimal attack, analysis of its efficiency and constraints. We derive bounds on the effectiveness of a poisoning attack against centroid anomaly under different conditions: bounded and unbounded percentage of traffic, and bounded false positive rate. Our bounds show that whereas a poisoning attack can be effectively staged in the unconstrained case, it can be made arbitrarily difficult (a strict upper bound on the attacker's gain) if external constraints are properly used. Our experimental evaluation carried out on real HTTP and exploit traces confirms the tightness of our theoretical bounds and practicality of our protection mechanisms.

Cited by in corpus (17)

• Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning
• Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering
• Support Vector Machines under Adversarial Label Contamination
• Detection of Adversarial Training Examples in Poisoning Attacks through Anomaly Detection
• Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learning
• Towards Adversarial Malware Detection: Lessons Learned from PDF-based Attacks
• Poisoning Attacks with Generative Adversarial Nets
• Biometric Backdoors: A Poisoning Attack Against Unsupervised Template Updating
• Poisoning the Unlabeled Dataset of Semi-Supervised Learning
• Poisoning and Backdooring Contrastive Learning
• Shielding Collaborative Learning: Mitigating Poisoning Attacks through Client-Side Detection
• A Survey on Resilient Machine Learning
• Security and Privacy for Artificial Intelligence: Opportunities and Challenges
• Regularisation Can Mitigate Poisoning Attacks: A Novel Analysis Based on Multiobjective Bilevel Optimisation
• Intelligent Systems Design for Malware Classification Under Adversarial Conditions
• Detecting Behavioral and Structural Anomalies in MediaCloud Applications
• Framework for Multi-task Multiple Kernel Learning and Applications in Genome Analysis