Group Distributionally Robust Knowledge Distillation

Konstantinos Vilouras, Xiao Liu, Pedro Sanchez +2

Knowledge distillation enables fast and effective transfer of features learned from a bigger model to a smaller one. However, distillation objectives are susceptible to sub-populat…