Using Early Readouts to Mediate Featural Bias in Distillation

Rishabh Tiwari, Durga Sivasubramanian, Anmol Mekala +2

Deep networks tend to learn spurious feature-label correlations in real-world supervised learning tasks. This vulnerability is aggravated in distillation, where a student model may…