SchoenbAt: Rethinking Attention with Polynomial basis

Yuhan Guo, Lizhong Ding, Yuwan Yang +1

Kernelized attention extends the attention mechanism by modeling sequence correlations through kernel functions, making significant progresses in optimizing attention. Under the gu…