A Smaller Transformer in Your Transformer

Dhananjay Tomar, Marius Aasan, Andreas Kleppe +1

Recent findings indicate that Vision Transformers settle into locally similar computational phases, implying a level of depthwise computational redundancy. However, existing method…