SOLAR: SVD-Optimized Lifelong Attention for Recommendation

Chenghao Zhang, Chao Feng, Yuanhao Pu +8

Attention mechanism remains the defining operator in Transformers since it provides expressive global credit assignment, yet its $O(N^{2}d)$ time and memory cost in sequence length…