HARD: Hard Augmentations for Robust Distillation

Arne F. Nix, Max F. Burg, Fabian H. Sinz

Knowledge distillation (KD) is a simple and successful method to transfer knowledge from a teacher to a student model solely based on functional activity. However, current KD has a…