Toward Understanding Adversarial Distillation: Why Robust Teachers Fail

Hongsin Lee, Hye Won Chung

Adversarial Distillation aims to enhance student robustness by guiding the student with a robust teacher's soft labels within the min-max adversarial training framework, yet its su…