Context-Aware Knowledge Distillation with Adaptive Weighting for Image Classification

Zhengda Li

Knowledge distillation (KD) is a widely used technique to transfer knowledge from a large teacher network to a smaller student model. Traditional KD uses a fixed balancing factor a…