Adversarially Robust Neural Architectures

Abstract

Deep Neural Networks (DNNs) are vulnerable to adversarial attacks. Existing methods are devoted to developing various robust training strategies or regularizations to update the weights of the neural network. But beyond the weights, the overall structure and information flow in the network are explicitly determined by the neural architecture, which remains unexplored. This paper thus aims to improve the adversarial robustness of the network from the architecture perspective. We explore the relationship among adversarial robustness, Lipschitz constant, and architecture parameters and show that an appropriate constraint on architecture parameters could reduce the Lipschitz constant to further improve the robustness. The importance of architecture parameters could vary from operation to operation or connection to connection. We approximate the Lipschitz constant of the entire network through a univariate log-normal distribution, whose mean and variance are related to architecture parameters. The confidence can be fulfilled through formulating a constraint on the distribution parameters based on the cumulative function. Compared with adversarially trained neural architectures searched by various NAS algorithms as well as efficient human-designed models, our algorithm empirically achieves the best performance among all the models under various attacks on different datasets.

arXiv comment13 pages, 5 figures, 8 tables

References in corpus (23)

• Neural Machine Translation by Jointly Learning to Align and Translate
• Explaining and Harnessing Adversarial Examples
• Intriguing properties of neural networks
• Neural Architecture Search with Reinforcement Learning
• Densely Connected Convolutional Networks
• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Rethinking the Value of Network Pruning
• Theoretically Principled Trade-off between Robustness and Accuracy
• Efficient Neural Architecture Search via Parameter Sharing
• SMASH: One-Shot Model Architecture Search through HyperNetworks
• Adversarial Machine Learning at Scale
• PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examples
• Generating Adversarial Examples with Adversarial Networks
• Spectral Norm Regularization for Improving the Generalizability of Deep Learning
• Improving Adversarial Robustness via Promoting Ensemble Diversity
• Progressive Neural Architecture Search
• DeepFool: a simple and accurate method to fool deep neural networks
• Adversarial Examples for Semantic Segmentation and Object Detection
• Universal adversarial perturbations
• Feature Denoising for Improving Adversarial Robustness
• Extending Defensive Distillation
• L2-Nonexpansive Neural Networks

Cited by in corpus (9)

• MedViT: A Robust Vision Transformer for Generalized Medical Image Classification
• Robust Pre-Training by Adversarial Contrastive Learning
• Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks
• Neural Architecture Dilation for Adversarial Robustness
• Towards Robust Vision Transformer
• Discovering Robust Convolutional Architecture at Targeted Capacity: A Multi-Shot Approach
• FlatNAS: optimizing Flatness in Neural Architecture Search for Out-of-Distribution Robustness
• Effective, Efficient and Robust Neural Architecture Search
• Learning Diverse-Structured Networks for Adversarial Robustness