Understanding Adversarial Training: Increasing Local Stability of Neural Nets through Robust Optimization

Abstract

We propose a general framework for increasing local stability of Artificial Neural Nets (ANNs) using Robust Optimization (RO). We achieve this through an alternating minimization-maximization procedure, in which the loss of the network is minimized over perturbed examples that are generated at each parameter update. We show that adversarial training of ANNs is in fact robustification of the network optimization, and that our proposed framework generalizes previous approaches for increasing local stability of ANNs. Experimental results reveal that our approach increases the robustness of the network to existing adversarial examples, while making it harder to generate new ones. Furthermore, our algorithm improves the accuracy of the network also on the original test data.

References in corpus (5)

• Explaining and Harnessing Adversarial Examples
• ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models
• Towards Deep Neural Network Architectures Robust to Adversarial Examples
• Parseval Networks: Improving Robustness to Adversarial Examples
• Deep Roto-Translation Scattering for Object Classification

Cited by in corpus (128)

• Towards Deep Learning Models Resistant to Adversarial Attacks
• DeepXplore: Automated Whitebox Testing of Deep Learning Systems
• Theoretically Principled Trade-off between Robustness and Accuracy
• On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima
• Optimization Problems for Machine Learning: A Survey
• A Survey on ChatGPT: AI-Generated Contents, Challenges, and Solutions
• DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Cars
• Measuring the tendency of CNNs to Learn Surface Statistical Regularities
• On Adversarial Examples for Character-Level Neural Machine Translation
• Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey
• Houdini: Fooling Deep Structured Prediction Models
• Adversarially Robust Distillation
• Robustness of classifiers: from adversarial to random noise
• The Robust Manifold Defense: Adversarial Training using Generative Models
• Rademacher Complexity for Adversarially Robust Generalization
• Interpreting and Improving Adversarial Robustness of Deep Neural Networks with Neuron Sensitivity
• Adversarial Examples that Fool Detectors
• Deceiving End-to-End Deep Learning Malware Detectors using Adversarial Examples
• Blocking Transferability of Adversarial Examples in Black-Box Learning Systems
• Generative Adversarial Trainer: Defense to Adversarial Perturbations with GAN
• Adversarial Training against Location-Optimized Adversarial Patches
• A Closer Look at Accuracy vs. Robustness
• A Simple Explanation for the Existence of Adversarial Examples with Small Hamming Distance
• Boosting Fast Adversarial Training with Learnable Adversarial Initialization
• Testing and verification of neural-network-based safety-critical control software: A systematic literature review
• Robust Quantum Control in Games: an Adversarial Learning Approach
• Standard detectors aren't (currently) fooled by physical adversarial stop signs
• Adversarial Examples in Modern Machine Learning: A Review
• Defending against Adversarial Images using Basis Functions Transformations
• Adversarial Examples - A Complete Characterisation of the Phenomenon
• Understanding and Improving Fast Adversarial Training
• Recent Advances in Adversarial Training for Adversarial Robustness
• Detecting Adversarial Samples for Deep Neural Networks through Mutation Testing
• Provably Robust Boosted Decision Stumps and Trees against Adversarial Attacks
• Daedalus: Breaking Non-Maximum Suppression in Object Detection via Adversarial Examples
• Large batch size training of neural networks with adversarial training and second-order information
• AI Safety for Everyone
• Towards a Robust and Trustworthy Machine Learning System Development: An Engineering Perspective
• The Curious Case of Adversarially Robust Models: More Data Can Help, Double Descend, or Hurt Generalization
• DLA: Dense-Layer-Analysis for Adversarial Example Detection
• Attention Meets Perturbations: Robust and Interpretable Attention with Adversarial Training
• Increasing the Confidence of Deep Neural Networks by Coverage Analysis
• Deep Learning for Scene Classification: A Survey
• Constrained Learning with Non-Convex Losses
• When Explainability Meets Adversarial Learning: Detecting Adversarial Examples using SHAP Signatures
• Fooling a Real Car with Adversarial Traffic Signs
• More Data Can Expand the Generalization Gap Between Adversarially Robust and Standard Models
• HashTran-DNN: A Framework for Enhancing Robustness of Deep Neural Networks against Adversarial Malware Samples
• Improving adversarial robustness of deep neural networks by using semantic information
• On the effectiveness of adversarial training against common corruptions
• MixUp as Directional Adversarial Training
• ConvNets and ImageNet Beyond Accuracy: Understanding Mistakes and Uncovering Biases
• FenceBox: A Platform for Defeating Adversarial Examples with Data Augmentation Techniques
• Adversarial Examples Detection in Deep Networks with Convolutional Filter Statistics
• Ensemble Robustness and Generalization of Stochastic Deep Learning Algorithms
• Mitigating Advanced Adversarial Attacks with More Advanced Gradient Obfuscation Techniques
• Adversarial Attacks on Monocular Depth Estimation
• Robust Multilingual Part-of-Speech Tagging via Adversarial Training
• Orchestrating the Development Lifecycle of Machine Learning-Based IoT Applications: A Taxonomy and Survey
• Online Robustness Training for Deep Reinforcement Learning
• Improving Network Robustness against Adversarial Attacks with Compact Convolution
• AdvSPADE: Realistic Unrestricted Attacks for Semantic Segmentation
• Adversarial Extreme Multi-label Classification
• Understanding and Improvement of Adversarial Training for Network Embedding from an Optimization Perspective
• Concise Explanations of Neural Networks using Adversarial Training
• Adversarial Examples on Object Recognition: A Comprehensive Survey
• Google's Cloud Vision API Is Not Robust To Noise
• ATHENA: A Framework based on Diverse Weak Defenses for Building Adversarial Defense
• On the Generalization Properties of Adversarial Training
• Calibrated Surrogate Losses for Adversarially Robust Classification
• Convergence and Margin of Adversarial Training on Separable Data
• Testing the Robustness of AutoML Systems
• Gradient-Based Adversarial Training on Transformer Networks for Detecting Check-Worthy Factual Claims
• Adversarial Perturbations Prevail in the Y-Channel of the YCbCr Color Space
• Ulixes: Facial Recognition Privacy with Adversarial Machine Learning
• Generative Dynamic Patch Attack
• A Deep-learning-based Method for PIR-based Multi-person Localization
• Exploring the Vulnerability of Deep Neural Networks: A Study of Parameter Corruption
• Improving Robustness of Jet Tagging Algorithms with Adversarial Training
• SOAR: Second-Order Adversarial Regularization
• Defending against Adversarial Attack towards Deep Neural Networks via Collaborative Multi-task Training
• Semantics Preserving Adversarial Learning
• AI Enabling Technologies: A Survey
• The coupling effect of Lipschitz regularization in deep neural networks
• Lipschitz Networks and Distributional Robustness
• Generalised Lipschitz Regularisation Equals Distributional Robustness
• Analysis and Improvement of Adversarial Training in DQN Agents With Adversarially-Guided Exploration (AGE)
• Robust and Stable Black Box Explanations
• Exploring the Hyperparameter Landscape of Adversarial Robustness
• A Data Augmentation-based Defense Method Against Adversarial Attacks in Neural Networks
• Adversarially Robust Estimate and Risk Analysis in Linear Regression
• Detecting Adversarial Perturbations with Saliency
• ATRO: Adversarial Training with a Rejection Option
• Are Adversarial Examples Created Equal? A Learnable Weighted Minimax Risk for Robustness under Non-uniform Attacks
• Improve Adversarial Robustness via Weight Penalization on Classification Layer
• Mitigating Adversarial Attacks by Distributing Different Copies to Different Users
• Feature-Filter: Detecting Adversarial Examples through Filtering off Recessive Features
• Mitigating Gradient-based Adversarial Attacks via Denoising and Compression
• EagleEye: Attack-Agnostic Defense against Adversarial Inputs (Technical Report)
• Probably Approximately Correct Constrained Learning
• Latent Adversarial Defence with Boundary-guided Generation
• Vax-a-Net: Training-time Defence Against Adversarial Patch Attacks
• Concurrent Adversarial Learning for Large-Batch Training
• Adequacy of the Gradient-Descent Method for Classifier Evasion Attacks
• Certified Defense via Latent Space Randomized Smoothing with Orthogonal Encoders
• Ensemble Noise Simulation to Handle Uncertainty about Gradient-based Adversarial Attacks
• SPADE: A Spectral Method for Black-Box Adversarial Robustness Evaluation
• Towards Speeding up Adversarial Training in Latent Spaces
• Dynamic Efficient Adversarial Training Guided by Gradient Magnitude
• Towards Robust Pattern Recognition: A Review
• Enabling Cyberattack-Resilient Load Forecasting through Adversarial Machine Learning
• Unifying Bilateral Filtering and Adversarial Training for Robust Neural Networks
• Exploring Frequency Domain Interpretation of Convolutional Neural Networks
• Defending Adversarial Attacks via Semantic Feature Manipulation
• Orthogonal Deep Models As Defense Against Black-Box Attacks
• Adversarial attacks on neural networks through canonical Riemannian foliations
• Towards Optimal Randomized Strategies in Adversarial Example Game
• Adversarial Machine Learning for Cybersecurity and Computer Vision: Current Developments and Challenges
• Learning Models for Actionable Recourse
• Defense-friendly Images in Adversarial Attacks: Dataset and Metrics for Perturbation Difficulty
• Context-Aware Image Denoising with Auto-Threshold Canny Edge Detection to Suppress Adversarial Perturbation
• Adversarial Ranking Attack and Defense
• Attribution of Gradient Based Adversarial Attacks for Reverse Engineering of Deceptions
• TEAM: An Taylor Expansion-Based Method for Generating Adversarial Examples
• Beyond Categorical Label Representations for Image Classification
• On educating machines
• Enhancing Resilience of Deep Learning Networks by Means of Transferable Adversaries
• Adversarial Robustness of Deep Convolutional Candlestick Learner