Teacher-free Latent Self-distillation and Class-separable Representations for Lightweight IoT Attack Detection

Phai Vu Dinh, Quang Uy Nguyen, Thai Hoang Dinh +6

Knowledge distillation (KD) has been widely used to improve lightweight AI models by transferring soft-label knowledge from a large teacher model to a student model. However, exist…