Collaborative Teacher-Student Learning via Multiple Knowledge Transfer

Liyuan Sun, Jianping Gou, Baosheng Yu +2

Knowledge distillation (KD), as an efficient and effective model compression technique, has been receiving considerable attention in deep learning. The key to its success is to tra…