26 citations · 48 across the 10 of their papers we have counts for
Showing 2023Show all
3 papers · 1 filter
cs.CV2023
Cumulative Spatial Knowledge Distillation for Vision Transformers
Borui Zhao, Renjie Song, Jiajun Liang
Distilling knowledge from convolutional neural networks (CNNs) is a double-edged sword for vision transformers (ViTs). It boosts the performance since the image-friendly local-indu…
cs.CV2023
DOT: A Distillation-Oriented Trainer
Borui Zhao, Quan Cui, Renjie Song +1
Knowledge distillation transfers knowledge from a large model to a small one via task and distillation losses. In this paper, we observe a trade-off between task and distillation l…
cs.CV2023
Boosting Semi-Supervised Learning by Exploiting All Unlabeled Data
Yuhao Chen, Xin Tan, Borui Zhao +4
Semi-supervised learning (SSL) has attracted enormous attention due to its vast potential of mitigating the dependence on large labeled datasets. The latest methods (e.g., FixMatch…