29 citations · 34 across the 7 of their papers we have counts for
Showing 2021Show all
2 papers · 1 filter
cs.LG2021★ 3 cited
You Only Sample (Almost) Once: Linear Cost Self-Attention Via Bernoulli Sampling
Zhanpeng Zeng, Yunyang Xiong, Sathya N. Ravi +3
Transformer-based models are widely used in natural language processing (NLP). Central to the transformer model is the self-attention mechanism, which captures the interactions of…
cs.CL2021★ 29 cited
Nyströmformer: A Nyström-Based Algorithm for Approximating Self-Attention
Yunyang Xiong, Zhanpeng Zeng, Rudrasis Chakraborty +4
Transformers have emerged as a powerful tool for a broad range of natural language processing tasks. A key component that drives the impressive performance of Transformers is the s…