Feature Evolution and Migration during Vision Transformer Training

Joonas Järve, Halil Ibrahim Aysel, Tarun Khajuria +1

We present a novel view on feature evolution in Vision Transformers (ViTs) by visualizing the training process over two dimensions -- network depth (layer) and training time (epoch…