23 citations · 81 across the 9 of their papers we have counts for
4 papers · 1 filter
ViTKD: Practical Guidelines for ViT feature knowledge distillation
Zhendong Yang, Zhe Li, Ailing Zeng +3
Knowledge Distillation (KD) for Convolutional Neural Network (CNN) is extensively studied as a way to boost the performance of a small model. Recently, Vision Transformer (ViT) has…
Rethinking Knowledge Distillation via Cross-Entropy
Zhendong Yang, Zhe Li, Yuan Gong +4
Knowledge Distillation (KD) has developed extensively and boosted various tasks. The classical KD method adds the KD loss to the original cross-entropy (CE) loss. We try to decompo…
Masked Generative Distillation
Zhendong Yang, Zhe Li, Mingqi Shao +3
Knowledge distillation has been applied to various tasks successfully. The current distillation algorithm usually improves students' performance by imitating the output of the teac…
Transparent Shape from a Single View Polarization Image
Mingqi Shao, Chongkun Xia, Zhendong Yang +2
This paper presents a learning-based method for transparent surface estimation from a single view polarization image. Existing shape from polarization(SfP) methods have the difficu…