Embedded Knowledge Distillation in Depth-Level Dynamic Neural Network

Qi Zhao, Shuchang Lyu, Zhiwei Zhang +2

In real applications, different computation-resource devices need different-depth networks (e.g., ResNet-18/34/50) with high-accuracy. Usually, existing methods either design multi…