Soft Mixture-of-Recursions: Going Deeper with Recursive Vision Transformers

Sang In Lee, Jihun Park

Vision Transformers are typically scaled by increasing depth or width, at the cost of more unique parameters. We explore recursion as an alternative scaling axis that increases eff…