Zero-Shot Knowledge Distillation in Deep Networks

Gaurav Kumar Nayak, Konda Reddy Mopuri, Vaisakh Shaj +2

Knowledge distillation deals with the problem of training a smaller model (Student) from a high capacity source model (Teacher) so as to retain most of its performance. Existing ap…