Picket: Guarding Against Corrupted Data in Tabular Data during Learning and Inference
arXiv:2006.04730
Abstract
Data corruption is an impediment to modern machine learning deployments. Corrupted data can severely bias the learned model and can also lead to invalid inferences. We present, Picket, a simple framework to safeguard against data corruptions during both training and deployment of machine learning models over tabular data. For the training stage, Picket identifies and removes corrupted data points from the training data to avoid obtaining a biased model. For the deployment stage, Picket flags, in an online manner, corrupted query points to a trained machine learning model that due to noise will result in incorrect predictions. To detect corrupted data, Picket uses a self-supervised deep learning model for mixed-type tabular data, which we call PicketNet. To minimize the burden of deployment, learning a PicketNet model does not require any human-labeled data. Picket is designed as a plugin that can increase the robustness of any machine learning pipeline. We evaluate Picket on a diverse array of real-world data considering different corruption models that include systematic and adversarial noise during both training and testing. We show that Picket consistently safeguards against corrupted data during both training and deployment of various models ranging from SVMs to neural networks, beating a diverse array of competing methods that span from data quality validation models to robust outlier-detection models.
23 pages, 24 figures
References in corpus (10)
- Two-Stream Convolutional Networks for Action Recognition in Videos
- On Calibration of Modern Neural Networks
- Poisoning Attacks against Support Vector Machines
- Improving Adversarial Robustness via Promoting Ensemble Diversity
- Generative Poisoning Attack Method Against Neural Networks
- Rethinking Softmax Cross-Entropy Loss for Adversarial Robustness
- Stronger Data Poisoning Attacks Break Data Sanitization Defenses
- A New Defense Against Adversarial Images: Turning a Weakness into a Strength
- Enhancing Adversarial Defense by k-Winners-Take-All
- On Robust Mean Estimation under Coordinate-level Corruption