Heterogeneous Knowledge Distillation using Information Flow Modeling

Nikolaos Passalis, Maria Tzelepi, Anastasios Tefas

Knowledge Distillation (KD) methods are capable of transferring the knowledge encoded in a large and complex teacher into a smaller and faster student. Early methods were usually l…