Prototype-guided Cross-task Knowledge Distillation for Large-scale Models

Deng Li, Aming Wu, Yahong Han +1

Recently, large-scale pre-trained models have shown their advantages in many tasks. However, due to the huge computational complexity and storage requirements, it is challenging to…