How Wrong Am I? - Studying Adversarial Examples and their Impact on Uncertainty in Gaussian Process Machine Learning Models
arXiv:1711.06598
Abstract
Machine learning models are vulnerable to Adversarial Examples: minor perturbations to input samples intended to deliberately cause misclassification. Current defenses against adversarial examples, especially for Deep Neural Networks (DNN), are primarily derived from empirical developments, and their security guarantees are often only justified retroactively. Many defenses therefore rely on hidden assumptions that are subsequently subverted by increasingly elaborate attacks. This is not surprising: deep learning notoriously lacks a comprehensive mathematical framework to provide meaningful guarantees. In this paper, we leverage Gaussian Processes to investigate adversarial examples in the framework of Bayesian inference. Across different models and datasets, we find deviating levels of uncertainty reflect the perturbation introduced to benign samples by state-of-the-art attacks, including novel white-box attacks on Gaussian Processes. Our experiments demonstrate that even unoptimized uncertainty thresholds already reject adversarial examples in many scenarios. Comment: Thresholds can be broken in a modified attack, which was done in arXiv:1812.02606 (The limitations of model uncertainty in adversarial settings).
Reasoning incomplete. Fixed issue in arXiv:1812.02606 (The limitations of model uncertainty in adversarial settings)
References in corpus (9)
- Delving into Transferable Adversarial Examples and Black-box Attacks
- Stealing Machine Learning Models via Prediction APIs
- Deep Neural Networks as Gaussian Processes
- Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods
- Adversarial Examples: Attacks and Defenses for Deep Learning
- Simple Black-Box Adversarial Perturbations for Deep Networks
- Dropout Inference in Bayesian Neural Networks with Alpha-divergences
- Analyzing the Robustness of Nearest Neighbors to Adversarial Examples
- Statistical Meta-Analysis of Presentation Attacks for Secure Multibiometric Systems