Explicit and Implicit Knowledge Distillation via Unlabeled Data

Yuzheng Wang, Zuhao Ge, Zhaoyu Chen +4

Data-free knowledge distillation is a challenging model lightweight task for scenarios in which the original dataset is not available. Previous methods require a lot of extra compu…