A Coulomb Particle Model for Learning Kernel Attention in Transformers

Masoud Badiei Khuzani, Sharath Honnaiah, Atiq Islam +2

Randomized features provide a scalable approximation to kernel machines, but their performance depends strongly on the choice of feature distribution. We propose a particle-based m…