Adversarial Robustness of Supervised Sparse Coding

Abstract

Several recent results provide theoretical insights into the phenomena of adversarial examples. Existing results, however, are often limited due to a gap between the simplicity of the models studied and the complexity of those deployed in practice. In this work, we strike a better balance by considering a model that involves learning a representation while at the same time giving a precise generalization bound and a robustness certificate. We focus on the hypothesis class obtained by combining a sparsity-promoting encoder coupled with a linear classifier, and show an interesting interplay between the expressivity and stability of the (supervised) representation map and a notion of margin in the feature space. We bound the robust risk (to ${\ell }_2$-bounded perturbations) of hypotheses parameterized by dictionaries that achieve a mild encoder gap on training data. Furthermore, we provide a robustness certificate for end-to-end classification. We demonstrate the applicability of our analysis by computing certified accuracy on real data, and compare with other alternatives for certified robustness.

References in corpus (20)

• Adam: A Method for Stochastic Optimization
• Towards Deep Learning Models Resistant to Adversarial Attacks
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Task-Driven Dictionary Learning
• Towards Deep Neural Network Architectures Robust to Adversarial Examples
• Certified Defenses against Adversarial Examples
• Adversarially Robust Generalization Requires More Data
• Robustness May Be at Odds with Accuracy
• Fast Inference in Sparse Coding Algorithms with Applications to Object Recognition
• On Adaptive Attacks to Adversarial Example Defenses
• Working Locally Thinking Globally: Theoretical Guarantees for Convolutional Sparse Coding
• Adversarial examples from computational constraints
• Rademacher Complexity for Adversarially Robust Generalization
• Maximal Sparsity with Deep Networks?
• Sparse DNNs with Improved Adversarial Robustness
• Deep Residual Autoencoders for Expectation Maximization-inspired Dictionary Learning
• Theoretical Analysis of Adversarial Learning: A Minimax Approach
• Convergence and Margin of Adversarial Training on Separable Data
• Inductive Bias of Gradient Descent based Adversarial Training on Separable Data
• Adversarial Risk Bounds for Neural Networks through Sparsity based Compression

Cited by in corpus (3)

• On the Adversarial Robustness of LASSO Based Feature Selection
• Predify: Augmenting deep neural networks with brain-inspired predictive coding dynamics
• Improved Training of Sparse Coding Variational Autoencoder via Weight Normalization