Maximizing the Position Embedding for Vision Transformers with Global Average Pooling

Wonjun Lee, Bumsub Ham, Suhyun Kim

In vision transformers, position embedding (PE) plays a crucial role in capturing the order of tokens. However, in vision transformer structures, there is a limitation in the expre…