Cubit: Token Mixer with Kernel Ridge Regression

Chuanyang Zheng, Jiankai Sun, Yihang Gao +6

Since its introduction in 2017, the Transformer has become one of the most widely adopted architectures in modern deep learning. Despite extensive efforts to improve positional enc…