MetaDistiller: Network Self-Boosting via Meta-Learned Top-Down Distillation

Benlin Liu, Yongming Rao, Jiwen Lu +2

Knowledge Distillation (KD) has been one of the most popu-lar methods to learn a compact model. However, it still suffers from highdemand in time and computational resources caused…