DSRNA: Differentiable Search of Robust Neural Architectures
arXiv:2012.06122
Abstract
In deep learning applications, the architectures of deep neural networks are crucial in achieving high accuracy. Many methods have been proposed to search for high-performance neural architectures automatically. However, these searched architectures are prone to adversarial attacks. A small perturbation of the input data can render the architecture to change prediction outcomes significantly. To address this problem, we propose methods to perform differentiable search of robust neural architectures. In our methods, two differentiable metrics are defined to measure architectures' robustness, based on certified lower bound and Jacobian norm bound. Then we search for robust architectures by maximizing the robustness metrics. Different from previous approaches which aim to improve architectures' robustness in an implicit way: performing adversarial training and injecting random noise, our methods explicitly and directly maximize robustness metrics to harvest robust architectures. On CIFAR-10, ImageNet, and MNIST, we perform game-based evaluation and verification-based evaluation on the robustness of our methods. The experimental results show that our methods 1) are more robust to various norm-bound attacks than several robust NAS baselines; 2) are more accurate than baselines when there are no attacks; 3) have significantly higher certified lower bounds than baselines.
arXiv comment10 pages
References in corpus (19)
- Adam: A Method for Stochastic Optimization
- Neural Architecture Search with Reinforcement Learning
- ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models
- Towards Deep Learning Models Resistant to Adversarial Attacks
- Adversarial Attacks on Neural Networks for Graph Data
- Provable defenses against adversarial examples via the convex outer adversarial polytope
- Efficient Neural Architecture Search via Parameter Sharing
- Fast is better than free: Revisiting adversarial training
- Black-box Adversarial Attacks with Limited Queries and Information
- SNAS: Stochastic Neural Architecture Search
- Understanding and Robustifying Differentiable Architecture Search
- DARTS+: Improved Differentiable Architecture Search with Early Stopping
- Hierarchical Representations for Efficient Architecture Search
- Towards Fast Computation of Certified Robustness for ReLU Networks
- Query-Efficient Hard-label Black-box Attack:An Optimization-based Approach
- Stabilizing Differentiable Architecture Search via Perturbation-based Regularization
- Robust Learning with Jacobian Regularization
- Progressive DARTS: Bridging the Optimization Gap for NAS in the Wild
- Attacking Visual Language Grounding with Adversarial Examples: A Case Study on Neural Image Captioning