FAVOR#: Sharp Attention Kernel Approximations via New Classes of Positive Random Features

Valerii Likhosherstov, Krzysztof Choromanski, Avinava Dubey +3

The problem of efficient approximation of a linear operator induced by the Gaussian or softmax kernel is often addressed using random features (RFs) which yield an unbiased approxi…