13 citations · 13 across the 3 of their papers we have counts for
Showing cs.CLShow all
2 papers · 1 filter
cs.CL2025
Rectified Sparse Attention
Yutao Sun, Tianzhu Ye, Li Dong +6
Efficient long-sequence generation is a critical challenge for Large Language Models. While recent sparse decoding methods improve efficiency, they suffer from KV cache misalignmen…
cs.CL2024★ 13 cited
Differential Transformer
Tianzhu Ye, Li Dong, Yuqing Xia +4
Transformer tends to overallocate attention to irrelevant context. In this work, we introduce Diff Transformer, which amplifies attention to the relevant context while canceling no…