Macformer: Transformer with Random Maclaurin Feature Attention

Yuhan Guo, Lizhong Ding, Ye Yuan +1

Random feature attention (RFA) adopts random fourier feature (RFF) methods to approximate the softmax function, resulting in a linear time and space attention mechanism that enable…