924 citations · 2.7k across the 21 of their papers we have counts for
Showing cs.CLShow all
3 papers · 1 filter
cs.CL2023★ 5 cited
Toeplitz Neural Network for Sequence Modeling
Zhen Qin, Xiaodong Han, Weixuan Sun +6
Sequence modeling has important applications in natural language processing and computer vision. Recently, the transformer-based models have shown strong performance on various seq…
cs.CL2022
The Devil in Linear Transformer
Zhen Qin, XiaoDong Han, Weixuan Sun +4
Linear transformers aim to reduce the quadratic space-time complexity of vanilla transformers. However, they usually suffer from degraded performances on various tasks and corpus.…
cs.CL2022★ 65 cited
cosFormer: Rethinking Softmax in Attention
Zhen Qin, Weixuan Sun, Hui Deng +6
Transformer has shown great successes in natural language processing, computer vision, and audio processing. As one of its core components, the softmax attention helps to capture l…