230 citations · 505 across the 18 of their papers we have counts for
24 papers
The Devil in Linear Transformer
Zhen Qin, XiaoDong Han, Weixuan Sun +4
Linear transformers aim to reduce the quadratic space-time complexity of vanilla transformers. However, they usually suffer from degraded performances on various tasks and corpus.…
Linear Video Transformer with Feature Fixation
Kaiyue Lu, Zexiang Liu, Jianyuan Wang +8
Vision Transformers have achieved impressive performance in video classification, while suffering from the quadratic complexity caused by the Softmax attention mechanism. Some stud…
Locality Matters: A Locality-Biased Linear Attention for Automatic Speech Recognition
Jingyu Sun, Guiping Zhong, Dinghao Zhou +2
Conformer has shown a great success in automatic speech recognition (ASR) on many public benchmarks. One of its crucial drawbacks is the quadratic time-space complexity with respec…
Implicit Motion Handling for Video Camouflaged Object Detection
Xuelian Cheng, Huan Xiong, Deng-Ping Fan +4
We propose a new video camouflaged object detection (VCOD) framework that can exploit both short-term dynamics and long-term temporal consistency to detect camouflaged objects from…
cosFormer: Rethinking Softmax in Attention
Zhen Qin, Weixuan Sun, Hui Deng +6
Transformer has shown great successes in natural language processing, computer vision, and audio processing. As one of its core components, the softmax attention helps to capture l…
Memory-Free Generative Replay For Class-Incremental Learning
Xiaomeng Xin, Yiran Zhong, Yunzhong Hou +2
Regularization-based methods are beneficial to alleviate the catastrophic forgetting problem in class-incremental learning. With the absence of old task images, they often assume t…