Knowledge Distillation in Generations: More Tolerant Teachers Educate Better Students

Chenglin Yang, Lingxi Xie, Siyuan Qiao +1

We focus on the problem of training a deep neural network in generations. The flowchart is that, in order to optimize the target network (student), another network (teacher) with t…