Heterogeneous Generative Knowledge Distillation with Masked Image Modeling

Ziming Wang, Shumin Han, Xiaodi Wang +3

Small CNN-based models usually require transferring knowledge from a large model before they are deployed in computationally resource-limited edge devices. Masked image modeling (M…