Residual-as-Teacher: Mitigating Bias Propagation in Student--Teacher Estimation

Kakei Yamamoto, Martin J. Wainwright

We study statistical estimation in a student--teacher setting, where predictions from a pre-trained teacher are used to guide a student model. A standard approach is to train the s…