518 citations · 915 across the 27 of their papers we have counts for
Showing cs.CVShow all
3 papers · 1 filter
cs.CV2023★ 20 cited
Knowledge Diffusion for Distillation
Tao Huang, Yuan Zhang, Mingkai Zheng +4
The representation gap between teacher and student is an emerging topic in knowledge distillation (KD). To reduce the gap and improve the performance, current methods often resort…
cs.CV2023
Avatar Knowledge Distillation: Self-ensemble Teacher Paradigm with Uncertainty
Yuan Zhang, Weihua Chen, Yichen Lu +3
Knowledge distillation is an effective paradigm for boosting the performance of pocket-size model, especially when multiple teacher models are available, the student would break th…
cs.CV2022★ 7 cited
Masked Distillation with Receptive Tokens
Tao Huang, Yuan Zhang, Shan You +4
Distilling from the feature maps can be fairly effective for dense prediction tasks since both the feature discriminability and localization priors can be well transferred. However…