Learning Diverse-Structured Networks for Adversarial Robustness
arXiv:2102.01886
Abstract
In adversarial training (AT), the main focus has been the objective and optimizer while the model has been less studied, so that the models being used are still those classic ones in standard training (ST). Classic network architectures (NAs) are generally worse than searched NAs in ST, which should be the same in AT. In this paper, we argue that NA and AT cannot be handled independently, since given a dataset, the optimal NA in ST would be no longer optimal in AT. That being said, AT is time-consuming itself; if we directly search NAs in AT over large search spaces, the computation will be practically infeasible. Thus, we propose a diverse-structured network (DS-Net), to significantly reduce the size of the search space: instead of low-level operations, we only consider predefined atomic blocks, where an atomic block is a time-tested building block like the residual block. There are only a few atomic blocks and thus we can weight all atomic blocks rather than find the best one in a searched block of DS-Net, which is an essential trade-off between exploring diverse structures and exploiting the best structures. Empirical results demonstrate the advantages of DS-Net, i.e., weighting the atomic blocks.
arXiv commentICML2021, code: https://github.com/d12306/dsnet
References in corpus (26)
- On the difficulty of training Recurrent Neural Networks
- Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
- Theoretically Principled Trade-off between Robustness and Accuracy
- A Convergence Theory for Deep Learning via Over-Parameterization
- Provable defenses against adversarial examples via the convex outer adversarial polytope
- Certified Adversarial Robustness via Randomized Smoothing
- Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
- Fast is better than free: Revisiting adversarial training
- Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulation
- Improving Adversarial Robustness via Promoting Ensemble Diversity
- On the Convergence and Robustness of Adversarial Training
- Parseval Networks: Improving Robustness to Adversarial Examples
- Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networks
- Improving Adversarial Robustness of Ensembles with Diversity Training
- Smooth Adversarial Training
- Adversarially Robust Generalization Just Requires More Unlabeled Data
- Improving Adversarial Robustness via Channel-wise Activation Suppressing
- Adversarially Robust Neural Architectures
- Adversarial Distributional Training for Robust Deep Learning
- Curriculum Adversarial Training
- A Direct Approach to Robust Deep Learning Using Adversarial Networks
- Once-for-All Adversarial Training: In-Situ Tradeoff between Robustness and Accuracy for Free
- Evolving Robust Neural Architectures to Defend from Adversarial Attacks
- Defending against Whitebox Adversarial Attacks via Randomized Discretization
- Improving the Robustness of Deep Neural Networks via Adversarial Training with Triplet Loss
- DSRNA: Differentiable Search of Robust Neural Architectures