A Spectral View of Adversarially Robust Features

Abstract

Given the apparent difficulty of learning models that are robust to adversarial perturbations, we propose tackling the simpler problem of developing adversarially robust features. Specifically, given a dataset and metric of interest, the goal is to return a function (or multiple functions) that 1) is robust to adversarial perturbations, and 2) has significant variation across the datapoints. We establish strong connections between adversarially robust features and a natural spectral property of the geometry of the dataset and metric of interest. This connection can be leveraged to provide both robust features, and a lower bound on the robustness of any function that has significant variance across the dataset. Finally, we provide empirical evidence that the adversarially robust features given by this spectral approach can be fruitfully leveraged to learn a robust (and accurate) model.

arXiv commentNeurips 2018 (this version corrects a minor error in the proof of Theorem 5)

References in corpus (12)

• Adam: A Method for Stochastic Optimization
• Intriguing properties of neural networks
• Towards Deep Learning Models Resistant to Adversarial Attacks
• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Synthesizing Robust Adversarial Examples
• Certified Defenses against Adversarial Examples
• Adversarially Robust Generalization Requires More Data
• Robust Physical-World Attacks on Deep Learning Models
• Certifying Some Distributional Robustness with Principled Adversarial Training
• Adversarial vulnerability for any classifier
• Adversarial examples from computational constraints

Cited by in corpus (6)

• A Closer Look at Accuracy vs. Robustness
• Learning Certified Individually Fair Representations
• Adversarial Neural Pruning with Latent Vulnerability Suppression
• Learning Adversarially Robust Representations via Worst-Case Mutual Information Maximization
• GRASPEL: Graph Spectral Learning at Scale
• Adversarially Robust Low Dimensional Representations