Enhancing Data-Free Adversarial Distillation with Activation Regularization and Virtual Interpolation

Xiaoyang Qu, Jianzong Wang, Jing Xiao

Knowledge distillation refers to a technique of transferring the knowledge from a large learned model or an ensemble of learned models to a small model. This method relies on acces…