Training Crossroads for Recurrent Vision Transformers: Recurrence, Neural ODEs, and Deep Supervision

Grzegorz Gruszczynski, Pawel Olszowiec, Michal Byra +2

Vision Transformers (ViTs) achieve strong image-recognition performance, but their parameter count grows linearly with depth when each block is independently parameterized. Single-…