Adversarial Robustness through Local Linearization

Abstract

Adversarial training is an effective methodology for training deep neural networks that are robust against adversarial, norm-bounded perturbations. However, the computational cost of adversarial training grows prohibitively as the size of the model and number of input dimensions increase. Further, training against less expensive and therefore weaker adversaries produces models that are robust against weak attacks but break down under attacks that are stronger. This is often attributed to the phenomenon of gradient obfuscation; such models have a highly non-linear loss surface in the vicinity of training examples, making it hard for gradient-based attacks to succeed even though adversarial examples still exist. In this work, we introduce a novel regularizer that encourages the loss to behave linearly in the vicinity of the training data, thereby penalizing gradient obfuscation while encouraging robustness. We show via extensive experiments on CIFAR-10 and ImageNet, that models trained with our regularizer avoid gradient obfuscation and can be trained significantly faster than adversarial training. Using this regularizer, we exceed current state of the art and achieve 47% adversarial accuracy for ImageNet with l-infinity adversarial perturbations of radius 4/255 under an untargeted, strong, white-box attack. Additionally, we match state of the art results for CIFAR-10 at 8/255.

Cited by in corpus (74)

• Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
• Adversarial Weight Perturbation Helps Robust Generalization
• Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples
• RobustBench: a standardized adversarial robustness benchmark
• Fixing Data Augmentation to Improve Adversarial Robustness
• Smooth Adversarial Training
• Adversarial Training for Large Neural Language Models
• A Closer Look at Accuracy vs. Robustness
• Testing Robustness Against Unforeseen Adversaries
• MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radius
• Security and Privacy Issues in Deep Learning
• An Alternative Surrogate Loss for PGD-based Adversarial Testing
• Diversity can be Transferred: Output Diversification for White- and Black-box Attacks
• Understanding and Improving Fast Adversarial Training
• Recent Advances in Adversarial Training for Adversarial Robustness
• Regularizing Black-box Models for Improved Interpretability
• Witches' Brew: Industrial Scale Data Poisoning via Gradient Matching
• Do Wider Neural Networks Really Help Adversarial Robustness?
• Adversarial Distributional Training for Robust Deep Learning
• Geometry-aware Instance-reweighted Adversarial Training
• Globally-Robust Neural Networks
• Empirical Studies on the Properties of Linear Regions in Deep Neural Networks
• Learning Security Classifiers with Verified Global Robustness Properties
• Deep Learning for Scene Classification: A Survey
• Quantifying the Preferential Direction of the Model Gradient in Adversarial Training With Projected Gradient Descent
• Open Problems in Cooperative AI
• Intriguing properties of adversarial training at scale
• Exploring Memorization in Adversarial Training
• Attacking Adversarial Attacks as A Defense
• Bag of Tricks for Adversarial Training
• Learnable Boundary Guided Adversarial Training
• Playing it Safe: Adversarial Robustness with an Abstain Option
• Jacobian Adversarially Regularized Networks for Robustness
• Second Order Optimization for Adversarial Robustness and Interpretability
• SOAR: Second-Order Adversarial Regularization
• How and When Adversarial Robustness Transfers in Knowledge Distillation?
• Revisiting Adversarial Robustness Distillation: Robust Soft Labels Make Student Better
• Robust Adversarial Learning via Sparsifying Front Ends
• Toward Adversarial Robustness via Semi-supervised Robust Training
• Data Quality Matters For Adversarial Training: An Empirical Study
• Pyramid Adversarial Training Improves ViT Performance
• Understanding Catastrophic Overfitting in Adversarial Training
• Defending Against Image Corruptions Through Adversarial Augmentations
• Bridging the Gap Between Adversarial Robustness and Optimization Bias
• Are Adversarial Examples Created Equal? A Learnable Weighted Minimax Risk for Robustness under Non-uniform Attacks
• A Game Theoretic Analysis of Additive Adversarial Attacks and Defenses
• LAFEAT: Piercing Through Adversarial Defenses with Latent Features
• Adversarial Training for Commonsense Inference
• What it Thinks is Important is Important: Robustness Transfers through Input Gradients
• Smoothness Analysis of Adversarial Training
• Provable Robust Classification via Learned Smoothed Densities
• Adversarial Feature Desensitization
• Adversarial Examples in Multi-Layer Random ReLU Networks
• Targeted Adversarial Perturbations for Monocular Depth Prediction
• Feature Distillation With Guided Adversarial Contrastive Learning
• Adversarial Learning with Cost-Sensitive Classes
• THAT: Two Head Adversarial Training for Improving Robustness at Scale
• Utilizing Adversarial Targeted Attacks to Boost Adversarial Robustness
• A Study on the Uncertainty of Convolutional Layers in Deep Neural Networks
• A single gradient step finds adversarial examples on random two-layers neural networks
• Perceptual Deep Neural Networks: Adversarial Robustness through Input Recreation
• Reject Illegal Inputs with Generative Classifier Derived from Any Discriminative Classifier
• Virtual Data Augmentation: A Robust and General Framework for Fine-tuning Pre-trained Models
• Towards Zero-shot Commonsense Reasoning with Self-supervised Refinement of Language Models
• Adversarial training may be a double-edged sword
• A Closer Look at the Adversarial Robustness of Information Bottleneck Models
• Less is More: Feature Selection for Adversarial Robustness with Compressive Counter-Adversarial Attacks
• Training Efficiency and Robustness in Deep Learning
• Understanding Invariance via Feedforward Inversion of Discriminatively Trained Classifiers
• GraCIAS: Grassmannian of Corrupted Images for Adversarial Security
• Likelihood Landscapes: A Unifying Principle Behind Many Adversarial Defenses
• Adversarial Training with Stochastic Weight Average
• Adversarial Defense via Local Flatness Regularization
• Get Fooled for the Right Reason: Improving Adversarial Robustness through a Teacher-guided Curriculum Learning Approach