Augmenting Knowledge Distillation With Peer-To-Peer Mutual Learning For Model Compression

Usma Niyaz, Deepti R. Bathula

Knowledge distillation (KD) is an effective model compression technique where a compact student network is taught to mimic the behavior of a complex and highly trained teacher netw…