ReViT: Enhancing Vision Transformers Feature Diversity with Attention Residual Connections

Anxhelo Diko, Danilo Avola, Marco Cascio +1

Vision Transformer (ViT) self-attention mechanism is characterized by feature collapse in deeper layers, resulting in the vanishing of low-level visual features. However, such feat…