Drawing Robust Scratch Tickets: Subnetworks with Inborn Robustness Are Found within Randomly Initialized Networks

Abstract

Deep Neural Networks (DNNs) are known to be vulnerable to adversarial attacks, i.e., an imperceptible perturbation to the input can mislead DNNs trained on clean images into making erroneous predictions. To tackle this, adversarial training is currently the most effective defense method, by augmenting the training set with adversarial samples generated on the fly. Interestingly, we discover for the first time that there exist subnetworks with inborn robustness, matching or surpassing the robust accuracy of the adversarially trained networks with comparable model sizes, within randomly initialized networks without any model training, indicating that adversarial training on model weights is not indispensable towards adversarial robustness. We name such subnetworks Robust Scratch Tickets (RSTs), which are also by nature efficient. Distinct from the popular lottery ticket hypothesis, neither the original dense networks nor the identified RSTs need to be trained. To validate and understand this fascinating finding, we further conduct extensive experiments to study the existence and properties of RSTs under different models, datasets, sparsity patterns, and attacks, drawing insights regarding the relationship between DNNs' robustness and their initialization/overparameterization. Furthermore, we identify the poor adversarial transferability between RSTs of different sparsity ratios drawn from the same randomly initialized dense network, and propose a Random RST Switch (R2S) technique, which randomly switches between different RSTs, as a novel defense method built on top of RSTs. We believe our findings about RSTs have opened up a new perspective to study model robustness and extend the lottery ticket hypothesis.

Accepted at NeurIPS 2021

References in corpus (33)

• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
• Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks
• ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models
• Towards Deep Learning Models Resistant to Adversarial Attacks
• The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
• A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
• Rethinking the Value of Network Pruning
• Certified Adversarial Robustness via Randomized Smoothing
• Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
• Fast is better than free: Revisiting adversarial training
• Adversarial Attacks and Defences: A Survey
• Countering Adversarial Images using Input Transformations
• Detecting Adversarial Samples from Artifacts
• PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examples
• Prior Convictions: Black-Box Adversarial Attacks with Bandits and Priors
• Mitigating Adversarial Effects Through Randomization
• Comparing Rewinding and Fine-tuning in Neural Network Pruning
• Simple Black-box Adversarial Attacks
• advertorch v0.1: An Adversarial Robustness Toolbox based on PyTorch
• On Adaptive Attacks to Adversarial Example Defenses
• Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLP
• Deep $k$-Means: Re-Training and Parameter Sharing with Harder Cluster Assignments for Compressing Deep Convolutions
• Proving the Lottery Ticket Hypothesis: Pruning is All You Need
• Drawing Early-Bird Tickets: Towards More Efficient Training of Deep Networks
• Winning the Lottery with Continuous Sparsification
• When BERT Plays the Lottery, All Tickets Are Winning
• Defensive Quantization: When Efficiency Meets Robustness
• The Difficulty of Training Sparse Neural Networks
• EarlyBERT: Efficient BERT Training via Early-bird Lottery Tickets
• Robust Sparse Regularization: Simultaneously Optimizing Neural Network Robustness and Compactness
• Online Adversarial Purification based on Self-Supervision
• Towards Practical Lottery Ticket Hypothesis for Adversarial Training
• Train-by-Reconnect: Decoupling Locations of Weights from their Values