2 citations · 4 across the 20 of their papers we have counts for
3 papers · 2 filters
SCOUT: Toward Sub-Quadratic Attention via Segment Compression for Optimized Utility in Transformers
Aref Jafari, Yuhe Fan, Benyamin Jamialahmadi +3
Transformers have demonstrated strong performance across a wide range of sequence modeling tasks, but their quadratic attention complexity limits scalability to long sequences. Lin…
DTRNet: Dynamic Token Routing Network to Reduce Quadratic Costs in Transformers
Aman Sharma, Saeed Najafi, Parsa Farinneya +6
Transformers achieve state-of-the-art results across many tasks, but their uniform application of quadratic self-attention to every token at every layer makes them computationally…
ZETA: Leveraging Z-order Curves for Efficient Top-k Attention
Qiuhao Zeng, Jerry Huang, Peng Lu +4
Over recent years, the Transformer has become a fundamental building block for sequence modeling architectures. Yet at its core is the use of self-attention, whose memory and compu…