Unifying Adversarial Training Algorithms with Flexible Deep Data Gradient Regularization
arXiv:1601.07213
Abstract
Many previous proposals for adversarial training of deep neural nets have included di- rectly modifying the gradient, training on a mix of original and adversarial examples, using contractive penalties, and approximately optimizing constrained adversarial ob- jective functions. In this paper, we show these proposals are actually all instances of optimizing a general, regularized objective we call DataGrad. Our proposed DataGrad framework, which can be viewed as a deep extension of the layerwise contractive au- toencoder penalty, cleanly simplifies prior work and easily allows extensions such as adversarial training with multi-task cues. In our experiments, we find that the deep gra- dient regularization of DataGrad (which also has L1 and L2 flavors of regularization) outperforms alternative forms of regularization, including classical L1, L2, and multi- task, both on the original dataset as well as on adversarial sets. Furthermore, we find that combining multi-task optimization with DataGrad adversarial training results in the most robust performance.
References in corpus (7)
- Explaining and Harnessing Adversarial Examples
- Intriguing properties of neural networks
- Towards Deep Neural Network Architectures Robust to Adversarial Examples
- Distributional Smoothing with Virtual Adversarial Training
- Learning with a Strong Adversary
- A Unified Gradient Regularization Family for Adversarial Examples
- Improving Back-Propagation by Adding an Adversarial Gradient
Cited by in corpus (7)
- A trans-disciplinary review of deep learning research for water resources scientists
- Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey
- Robust Learning with Jacobian Regularization
- Learning Adversary-Resistant Deep Neural Networks
- Interpreting Adversarial Robustness: A View from Decision Surface in Input Space
- Using Non-invertible Data Transformations to Build Adversarial-Robust Neural Networks
- Adversary Resistant Deep Neural Networks with an Application to Malware Detection