Flow-based Spectral Kernel Learning for Nonstationary Attention

Zi Yang, Ying Li, Alejandro Lancho +3

Kernelized attention provides a useful perspective for understanding self-attention as a similarity-based aggregation mechanism, but existing approaches often rely on fixed or stat…