Boosting Adversarial Training with Hypersphere Embedding

Abstract

Adversarial training (AT) is one of the most effective defenses against adversarial attacks for deep learning models. In this work, we advocate incorporating the hypersphere embedding (HE) mechanism into the AT procedure by regularizing the features onto compact manifolds, which constitutes a lightweight yet effective module to blend in the strength of representation learning. Our extensive analyses reveal that AT and HE are well coupled to benefit the robustness of the adversarially trained models from several aspects. We validate the effectiveness and adaptability of HE by embedding it into the popular AT frameworks including PGD-AT, ALP, and TRADES, as well as the FreeAT and FastAT strategies. In the experiments, we evaluate our methods under a wide range of adversarial attacks on the CIFAR-10 and ImageNet datasets, which verifies that integrating HE can consistently enhance the model robustness for each AT framework with little extra computation.

arXiv commentNeurIPS 2020

References in corpus (35)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Ensemble Adversarial Training: Attacks and Defenses
• ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models
• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
• Theoretically Principled Trade-off between Robustness and Accuracy
• Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
• On Evaluating Adversarial Robustness
• Fast is better than free: Revisiting adversarial training
• Using Pre-Training Can Improve Model Robustness and Uncertainty
• Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
• Making Convolutional Networks Shift-Invariant Again
• Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers
• Improving Adversarial Robustness via Promoting Ensemble Diversity
• On the Convergence and Robustness of Adversarial Training
• Are Labels Required for Improving Adversarial Robustness?
• Adversarial Examples Are a Natural Consequence of Test Error in Noise
• Rethinking Softmax Cross-Entropy Loss for Adversarial Robustness
• Practical Black-Box Attacks against Machine Learning
• Image Synthesis with a Single (Robust) Classifier
• Deep Defense: Training DNNs with Improved Adversarial Robustness
• Improving the Generalization of Adversarial Training with Domain Adaptation
• Adversarially Robust Generalization Just Requires More Unlabeled Data
• Evaluating and Understanding the Robustness of Adversarial Logit Pairing
• Interpreting Adversarially Trained Convolutional Neural Networks
• First-order Adversarial Vulnerability of Neural Networks and Input Dimension
• Metric Learning for Adversarial Robustness
• An Alternative Surrogate Loss for PGD-based Adversarial Testing
• Overfitting in adversarially robust deep learning
• Max-Mahalanobis Linear Discriminant Analysis Networks
• Benchmarking Adversarial Robustness
• Bag of Tricks for Adversarial Training
• Improving Machine Reading Comprehension via Adversarial Training
• RayS: A Ray Searching Method for Hard-label Adversarial Attack
• Using Single-Step Adversarial Training to Defend Iterative Adversarial Examples
• Regularizers for Single-step Adversarial Training

Cited by in corpus (11)

• Large-Scale Adversarial Training for Vision-and-Language Representation Learning
• A Review of Adversarial Attack and Defense for Classification Methods
• Adversarial Examples Improve Image Recognition
• Do Wider Neural Networks Really Help Adversarial Robustness?
• A Closer Look at the Robustness of Vision-and-Language Pre-trained Models
• Voting based ensemble improves robustness of defensive models
• Adversarial Robustness under Long-Tailed Distribution
• HateProof: Are Hateful Meme Detection Systems really Robust?
• Adversarial Robustness through the Lens of Convolutional Filters
• Imbalanced Gradients: A Subtle Cause of Overestimated Adversarial Robustness
• Dual Head Adversarial Training