36 citations · 167 across the 25 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2022★ 28 cited
VAQF: Fully Automatic Software-Hardware Co-Design Framework for Low-Bit Vision Transformer
Mengshu Sun, Haoyu Ma, Guoliang Kang +5
The transformer architectures with attention mechanisms have obtained success in Nature Language Processing (NLP), and Vision Transformers (ViTs) have recently extended the applica…
cs.LG2021★ 19 cited
Undistillable: Making A Nasty Teacher That CANNOT teach students
Haoyu Ma, Tianlong Chen, Ting-Kuei Hu +3
Knowledge Distillation (KD) is a widely used technique to transfer knowledge from pre-trained teacher models to (usually more lightweight) student models. However, in certain situa…