9 citations · 26 across the 20 of their papers we have counts for
Showing 2022Show all
2 papers · 1 filter
cs.LG2022
CAB: Comprehensive Attention Benchmarking on Long Sequence Modeling
Jun Zhang, Shuyang Jiang, Jiangtao Feng +2
Transformer has achieved remarkable success in language, image, and speech processing. Recently, various efficient attention architectures have been proposed to improve transformer…
cs.LG2022★ 8 cited
Linear Complexity Randomized Self-attention Mechanism
Lin Zheng, Chong Wang, Lingpeng Kong
Recently, random feature attentions (RFAs) are proposed to approximate the softmax attention in linear time and space complexity by linearizing the exponential kernel. In this pape…