A New Training Framework for Deep Neural Network

Zhenyan Hou, Wenxuan Fan

Knowledge distillation is the process of transferring the knowledge from a large model to a small model. In this process, the small model learns the generalization ability of the l…