Knowledge Distillation with Adapted Weight

Sirong Wu, Xi Luo, Junjie Liu +1

Although large models have shown a strong capacity to solve large-scale problems in many areas including natural language and computer vision, their voluminous parameters are hard…