Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learning

Abstract

As machine learning becomes widely used for automated decisions, attackers have strong incentives to manipulate the results and models generated by machine learning algorithms. In this paper, we perform the first systematic study of poisoning attacks and their countermeasures for linear regression models. In poisoning attacks, attackers deliberately influence the training data to manipulate the results of a predictive model. We propose a theoretically-grounded optimization framework specifically designed for linear regression and demonstrate its effectiveness on a range of datasets and models. We also introduce a fast statistical attack that requires limited knowledge of the training process. Finally, we design a new principled defense method that is highly resilient against all poisoning attacks. We provide formal guarantees about its convergence and an upper bound on the effect of poisoning attacks when the defense is deployed. We evaluate extensively our attacks and defenses on three realistic datasets from health care, loan assessment, and real estate domains.

Preprint of the work accepted for publication at the 39th IEEE Symposium on Security and Privacy, San Francisco, CA, USA, May 21-23, 2018; Sept 28 '21 update: add citation to trimmed losses

References in corpus (8)

• Explaining and Harnessing Adversarial Examples
• Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning
• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• Poisoning Attacks against Support Vector Machines
• Is feature selection secure against training data poisoning?
• Data Poisoning Attacks on Factorization-Based Collaborative Filtering
• Security Analysis of Online Centroid Anomaly Detection
• Robust High Dimensional Sparse Regression and Matching Pursuit

Cited by in corpus (15)

• Security and Privacy Issues in Deep Learning
• Entangled Watermarks as a Defense against Model Extraction
• DeepView: Visualizing Classification Boundaries of Deep Neural Networks as Scatter Plots Using Discriminative Dimensionality Reduction
• PAC-learning in the presence of evasion adversaries
• Helen: Maliciously Secure Coopetitive Learning for Linear Models
• Adversarial Security Attacks and Perturbations on Machine Learning and Deep Learning Methods
• An Optimal Control View of Adversarial Machine Learning
• PAC-Learning for Strategic Classification
• Data Poisoning Attacks in Contextual Bandits
• Classification Auto-Encoder based Detector against Diverse Data Poisoning Attacks
• Oversight of Unsafe Systems via Dynamic Safety Envelopes
• Law and Adversarial Machine Learning
• Machine learning in physics: The pitfalls of poisoned training sets
• Machine learning pipeline for battery state of health estimation
• Widen The Backdoor To Let More Attackers In