Adversarial Defense via Data Dependent Activation Function and Total Variation Minimization

Abstract

We improve the robustness of Deep Neural Net (DNN) to adversarial attacks by using an interpolating function as the output activation. This data-dependent activation remarkably improves both the generalization and robustness of DNN. In the CIFAR10 benchmark, we raise the robust accuracy of the adversarially trained ResNet20 from $\sim 46\%$ to $\sim 69\%$ under the state-of-the-art Iterative Fast Gradient Sign Method (IFGSM) based adversarial attack. When we combine this data-dependent activation with total variation minimization on adversarial images and training data augmentation, we achieve an improvement in robust accuracy by 38.9$\%$ for ResNet56 under the strongest IFGSM attack. Furthermore, We provide an intuitive explanation of our defense by analyzing the geometry of the feature space.

arXiv comment17 pages, 6 figures

References in corpus (13)

• Adam: A Method for Stochastic Optimization
• Ensemble Adversarial Training: Attacks and Defenses
• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
• A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
• Delving into Transferable Adversarial Examples and Black-box Attacks
• Adversarial Perturbations Against Deep Neural Networks for Malware Classification
• Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey
• Foveation-based Mechanisms Alleviate Adversarial Examples
• Divide, Denoise, and Defend against Adversarial Attacks
• Deep Neural Nets with Interpolating Function as Output Activation
• Reinforcing Adversarial Robustness using Model Confidence Induced by Adversarial Training
• Error estimation of weighted nonlocal Laplacian on random point cloud

Cited by in corpus (6)

• Improving Adversarial Robustness via Channel-wise Activation Suppressing
• On Robustness of Neural Ordinary Differential Equations
• ResNets Ensemble via the Feynman-Kac Formalism to Improve Natural and Robust Accuracies
• Mathematical Analysis of Adversarial Attacks
• Adversarial Parameter Defense by Multi-Step Risk Minimization
• Graph Interpolating Activation Improves Both Natural and Robust Accuracies in Data-Efficient Deep Learning