22 citations · 72 across the 14 of their papers we have counts for
Showing 2024Show all
3 papers · 1 filter
cs.CL2024
Gated Slot Attention for Efficient Linear-Time Sequence Modeling
Yu Zhang, Songlin Yang, Ruijie Zhu +9
Linear attention Transformers and their gated variants, celebrated for enabling parallel training and efficient recurrent inference, still fall short in recall-intensive tasks comp…
cs.LG2024★ 3 cited
Parallelizing Linear Transformers with the Delta Rule over Sequence Length
Songlin Yang, Bailin Wang, Yu Zhang +2
Transformers with linear attention (i.e., linear transformers) and state-space models have recently been suggested as a viable linear-time alternative to transformers with softmax…
cs.CL2024★ 2 cited
Scalable MatMul-free Language Modeling
Rui-Jie Zhu, Yu Zhang, Steven Abreu +7
Large Language Models (LLMs) have fundamentally altered how we approach scaling in machine learning. However, these models pose substantial computational and memory challenges, pri…