Achieving Adversarial Robustness Requires An Active Teacher

Abstract

A new understanding of adversarial examples and adversarial robustness is proposed by decoupling the data generator and the label generator (which we call the teacher). In our framework, adversarial robustness is a conditional concept---the student model is not absolutely robust, but robust with respect to the teacher. Based on the new understanding, we claim that adversarial examples exist because the student cannot obtain sufficient information of the teacher from the training data. Various ways of achieving robustness is compared. Theoretical and numerical evidence shows that to efficiently attain robustness, a teacher that actively provides its information to the student may be necessary.

References in corpus (16)

• Distilling the Knowledge in a Neural Network
• Towards Deep Learning Models Resistant to Adversarial Attacks
• Provable defenses against adversarial examples via the convex outer adversarial polytope
• Countering Adversarial Images using Input Transformations
• Robustness May Be at Odds with Accuracy
• Out-of-Distribution Generalization via Risk Extrapolation (REx)
• Exploring the Landscape of Spatial Robustness
• Adversarial Robustness May Be at Odds With Simplicity
• Test-Time Training with Self-Supervision for Generalization under Distribution Shifts
• Failures of Gradient-Based Deep Learning
• Quantifying Perceptual Distortion of Adversarial Examples
• A Theoretical Framework for Robustness of (Deep) Classifiers against Adversarial Examples
• Perceptual Adversarial Robustness: Defense Against Unseen Threat Models
• Should Adversarial Attacks Use Pixel p-Norm?
• Statistical-Query Lower Bounds via Functional Gradients
• Over-parameterized Adversarial Training: An Analysis Overcoming the Curse of Dimensionality