Adversarial Parameter Defense by Multi-Step Risk Minimization

Abstract

Previous studies demonstrate DNNs' vulnerability to adversarial examples and adversarial training can establish a defense to adversarial examples. In addition, recent studies show that deep neural networks also exhibit vulnerability to parameter corruptions. The vulnerability of model parameters is of crucial value to the study of model robustness and generalization. In this work, we introduce the concept of parameter corruption and propose to leverage the loss change indicators for measuring the flatness of the loss basin and the parameter robustness of neural network parameters. On such basis, we analyze parameter corruptions and propose the multi-step adversarial corruption algorithm. To enhance neural networks, we propose the adversarial parameter defense algorithm that minimizes the average risk of multiple adversarial parameter corruptions. Experimental results show that the proposed algorithm can improve both the parameter robustness and accuracy of neural networks.

arXiv commentAccepted to Neural Networks. A substantial journal extension of our previous conference paper arXiv:2006.05620

References in corpus (15)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Explaining and Harnessing Adversarial Examples
• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• FreeLB: Enhanced Adversarial Training for Natural Language Understanding
• Stochastic Activation Pruning for Robust Adversarial Defense
• fairseq: A Fast, Extensible Toolkit for Sequence Modeling
• Sharpness-Aware Minimization for Efficiently Improving Generalization
• Stronger generalization bounds for deep nets via a compression approach
• You Only Propagate Once: Accelerating Adversarial Training via Maximal Principle
• Maximum-Entropy Adversarial Data Augmentation for Improved Generalization and Robustness
• Weight Poisoning Attacks on Pre-trained Models
• LCA: Loss Change Allocation for Neural Network Training
• Adversarial Defense via Data Dependent Activation Function and Total Variation Minimization
• Regularizing Neural Networks via Adversarial Model Perturbation
• Improved Corruption Robust Algorithms for Episodic Reinforcement Learning