Rethinking Query, Key, and Value Embedding in Vision Transformer under Tiny Model Constraints

Jaesin Ahn, Jiuk Hong, Jeongwoo Ju +1

A vision transformer (ViT) is the dominant model in the computer vision field. Despite numerous studies that mainly focus on dealing with inductive bias and complexity, there remai…