2 papers
cs.LG2025
SchoenbAt: Rethinking Attention with Polynomial basis
Yuhan Guo, Lizhong Ding, Yuwan Yang +1
Kernelized attention extends the attention mechanism by modeling sequence correlations through kernel functions, making significant progresses in optimizing attention. Under the gu…
cs.LG2024
Macformer: Transformer with Random Maclaurin Feature Attention
Yuhan Guo, Lizhong Ding, Ye Yuan +1
Random feature attention (RFA) adopts random fourier feature (RFF) methods to approximate the softmax function, resulting in a linear time and space attention mechanism that enable…