First-order Adversarial Vulnerability of Neural Networks and Input Dimension

Abstract

Over the past few years, neural networks were proven vulnerable to adversarial images: targeted but imperceptible image perturbations lead to drastically different predictions. We show that adversarial vulnerability increases with the gradients of the training objective when viewed as a function of the inputs. Surprisingly, vulnerability does not depend on network topology: for many standard network architectures, we prove that at initialization, the ${\ell }_1$-norm of these gradients grows as the square root of the input dimension, leaving the networks increasingly vulnerable with growing image size. We empirically show that this dimension dependence persists after either usual or robust training, but gets attenuated with higher regularization.

arXiv commentPaper previously called: "Adversarial Vulnerability of Neural Networks Increases with Input Dimension". 9 pages main text and references, 11 pages appendix, 14 figures

Cited by in corpus (27)

• Self-training with Noisy Student improves ImageNet classification
• Boosting Adversarial Training with Hypersphere Embedding
• Vulnerability of deep neural networks for detecting COVID-19 cases from chest X-ray images to universal adversarial attacks
• Batch Normalization is a Cause of Adversarial Vulnerability
• Walking on the Edge: Fast, Low-Distortion Adversarial Examples
• Understanding and Improving Fast Adversarial Training
• On the effectiveness of adversarial training against common corruptions
• Multitask Learning Strengthens Adversarial Robustness
• How to Learn a Useful Critic? Model-based Action-Gradient-Estimator Policy Optimization
• Towards Security Threats of Deep Learning Systems: A Survey
• Jacobian Adversarially Regularized Networks for Robustness
• Why Adversarial Reprogramming Works, When It Fails, and How to Tell the Difference
• SOAR: Second-Order Adversarial Regularization
• Shift Invariance Can Reduce Adversarial Robustness
• Recent Advances in Understanding Adversarial Robustness of Deep Neural Networks
• Toward Adversarial Robustness via Semi-supervised Robust Training
• Improve Generalization and Robustness of Neural Networks via Weight Scale Shifting Invariant Regularizations
• Bridging the Gap Between Adversarial Robustness and Optimization Bias
• White-Box Adversarial Defense via Self-Supervised Data Estimation
• An Empirical Study on the Relation between Network Interpretability and Adversarial Robustness
• Brittle Features May Help Anomaly Detection
• Who's Afraid of Adversarial Transferability?
• Analysis of Random Perturbations for Robust Convolutional Neural Networks
• Training Efficiency and Robustness in Deep Learning
• Improving Resistance to Adversarial Deformations by Regularizing Gradients
• Accumulative Poisoning Attacks on Real-time Data
• Adversarial Defense via Local Flatness Regularization