Showing 2024Show all
2 papers · 1 filter
cs.CV2024
Neural Collapse Inspired Knowledge Distillation
Shuoxi Zhang, Zijian Song, Kun He
Existing knowledge distillation (KD) methods have demonstrated their ability in achieving student network performance on par with their teachers. However, the knowledge gap between…
cs.CV2024
You Only Need Less Attention at Each Stage in Vision Transformers
Shuoxi Zhang, Hanpeng Liu, Stephen Lin +1
The advent of Vision Transformers (ViTs) marks a substantial paradigm shift in the realm of computer vision. ViTs capture the global information of images through self-attention mo…