4 papers
Copy the Same, Distill the Difference: Initializing Linear Vision Transformers
Huaiyuan Qin, Muli Yang, Gabriel James Goenawan +9
Linear Vision Transformers (ViTs) are designed to replace the attention in Softmax ViTs with the linear-complexity attention operator for more efficient token routing, but they req…
Robust Fuzzy Multi-view Learning under View Conflict
Siyuan Duan, Yuan Sun, Dezhong Peng +3
Trusted multi-view classification aims to deliver reliable fusion for accurate predictions and has recently attracted substantial attention in both academia and industry. However,…
Attention Transfer Is Not Universally Effective for Vision Transformers
Huaiyuan Qin, Muli Yang, Gabriel James Goenawan +4
A recent work shows that Attention Transfer, which transfers only the attention patterns from a pre-trained teacher Vision Transformer (ViT) to a randomly initialized standard stud…
Beyond Loss Values: Robust Dynamic Pruning via Loss Trajectory Alignment
Huaiyuan Qin, Muli Yang, Gabriel James Goenawan +5
Existing dynamic data pruning methods often fail under noisy-label settings, as they typically rely on per-sample loss as the ranking criterion. This could mistakenly lead to prese…