Black-box Few-shot Knowledge Distillation

Dang Nguyen, Sunil Gupta, Kien Do +1

Knowledge distillation (KD) is an efficient approach to transfer the knowledge from a large "teacher" network to a smaller "student" network. Traditional KD methods require lots of…