65 citations · 68 across the 4 of their papers we have counts for
4 papers
The Devil in Linear Transformer
Zhen Qin, XiaoDong Han, Weixuan Sun +4
Linear transformers aim to reduce the quadratic space-time complexity of vanilla transformers. However, they usually suffer from degraded performances on various tasks and corpus.…
Linear Video Transformer with Feature Fixation
Kaiyue Lu, Zexiang Liu, Jianyuan Wang +8
Vision Transformers have achieved impressive performance in video classification, while suffering from the quadratic complexity caused by the Softmax attention mechanism. Some stud…
cosFormer: Rethinking Softmax in Attention
Zhen Qin, Weixuan Sun, Hui Deng +6
Transformer has shown great successes in natural language processing, computer vision, and audio processing. As one of its core components, the softmax attention helps to capture l…
3D Guided Weakly Supervised Semantic Segmentation
Weixuan Sun, Jing Zhang, Nick Barnes
Pixel-wise clean annotation is necessary for fully-supervised semantic segmentation, which is laborious and expensive to obtain. In this paper, we propose a weakly supervised 2D se…