Knowledge Transfer via Dense Cross-Layer Mutual-Distillation

Anbang Yao, Dawei Sun

Knowledge Distillation (KD) based methods adopt the one-way Knowledge Transfer (KT) scheme in which training a lower-capacity student network is guided by a pre-trained high-capaci…