31 citations · 83 across the 14 of their papers we have counts for
Showing 2023Show all
3 papers · 1 filter
cs.CV2023★ 31 cited
One-for-All: Bridge the Gap Between Heterogeneous Architectures in Knowledge Distillation
Zhiwei Hao, Jianyuan Guo, Kai Han +4
Knowledge distillation~(KD) has proven to be a highly effective approach for enhancing model performance through a teacher-student training scheme. However, most existing distillat…
cs.CV2023★ 1 cited
DeViT: Decomposing Vision Transformers for Collaborative Inference in Edge Devices
Guanyu Xu, Zhiwei Hao, Yong Luo +3
Recent years have witnessed the great success of vision transformer (ViT), which has achieved state-of-the-art performance on multiple computer vision benchmarks. However, ViT mode…
cs.CV2023★ 5 cited
VanillaKD: Revisit the Power of Vanilla Knowledge Distillation from Small Scale to Large Scale
Zhiwei Hao, Jianyuan Guo, Kai Han +3
The tremendous success of large models trained on extensive datasets demonstrates that scale is a key ingredient in achieving superior results. Therefore, the reflection on the rat…