Linearizing Vision Transformer with Test-Time Training

Yining Li, Dongchen Han, Zeyu Liu +3

While linear-complexity attention mechanisms offer a promising alternative to Softmax attention for overcoming the quadratic bottleneck, training such models from scratch remains p…