Adversarial robustness via robust low rank representations

Abstract

Adversarial robustness measures the susceptibility of a classifier to imperceptible perturbations made to the inputs at test time. In this work we highlight the benefits of natural low rank representations that often exist for real data such as images, for training neural networks with certified robustness guarantees. Our first contribution is for certified robustness to perturbations measured in ${\ell }_2$ norm. We exploit low rank data representations to provide improved guarantees over state-of-the-art randomized smoothing-based approaches on standard benchmark datasets such as CIFAR-10 and CIFAR-100. Our second contribution is for the more challenging setting of certified robustness to perturbations measured in ${\ell }_{\infty }$ norm. We demonstrate empirically that natural low rank representations have inherent robustness properties, that can be leveraged to provide significantly better guarantees for certified robustness to ${\ell }_{\infty }$ perturbations in those representations. Our certificate of ${\ell }_{\infty }$ robustness relies on a natural quantity involving the $\infty \to 2$ matrix operator norm associated with the representation, to translate robustness guarantees from ${\ell }_2$ to ${\ell }_{\infty }$ perturbations. A key technical ingredient for our certification guarantees is a fast algorithm with provable guarantees based on the multiplicative weights update method to provide upper bounds on the above matrix norm. Our algorithmic guarantees improve upon the state of the art for this problem, and may be of independent interest.

fixed a bug in the proof of Proposition B.2

References in corpus (20)

• Scikit-learn: Machine Learning in Python
• Towards Deep Learning Models Resistant to Adversarial Attacks
• Theoretically Principled Trade-off between Robustness and Accuracy
• Provable defenses against adversarial examples via the convex outer adversarial polytope
• Certified Adversarial Robustness via Randomized Smoothing
• On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models
• Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers
• Semidefinite relaxations for certifying robustness to adversarial examples
• Certified Adversarial Robustness with Additive Noise
• Efficient Formal Safety Analysis of Neural Networks
• Training verified learners with learned verifiers
• A Closer Look at Accuracy vs. Robustness
• Provable Robustness of ReLU networks via Maximization of Linear Regions
• ME-Net: Towards Effective Adversarial Robustness with Matrix Estimation
• Curse of Dimensionality on Randomized Smoothing for Certifiable Robustness
• Randomized Smoothing of All Shapes and Sizes
• On Robustness to Adversarial Examples and Polynomial Optimization
• Random Smoothing Might be Unable to Certify ${\ell }_{\infty }$ Robustness for High-Dimensional Images
• Adversarially Robust Low Dimensional Representations
• Inapproximability of Matrix $p\to q$ Norms

Cited by in corpus (1)

• SPECTRE: Defending Against Backdoor Attacks Using Robust Statistics