EViT: An Eagle Vision Transformer with Bi-Fovea Self-Attention

Abstract

Owing to advancements in deep learning technology, Vision Transformers (ViTs) have demonstrated impressive performance in various computer vision tasks. Nonetheless, ViTs still face some challenges, such as high computational complexity and the absence of desirable inductive biases. To alleviate these issues, {the potential advantages of combining eagle vision with ViTs are explored. We summarize a Bi-Fovea Visual Interaction (BFVI) structure inspired by the unique physiological and visual characteristics of eagle eyes. A novel Bi-Fovea Self-Attention (BFSA) mechanism and Bi-Fovea Feedforward Network (BFFN) are proposed based on this structural design approach, which can be used to mimic the hierarchical and parallel information processing scheme of the biological visual cortex, enabling networks to learn feature representations of targets in a coarse-to-fine manner. Furthermore, a Bionic Eagle Vision (BEV) block is designed as the basic building unit based on the BFSA mechanism and BFFN. By stacking BEV blocks, a unified and efficient family of pyramid backbone networks called Eagle Vision Transformers (EViTs) is developed. Experimental results show that EViTs exhibit highly competitive performance in various computer vision tasks, such as image classification, object detection and semantic segmentation. Compared with other approaches, EViTs have significant advantages, especially in terms of performance and computational efficiency. Code is available at https://github.com/nkusyl/EViT

This work has been submitted to the IEEE for possible publication

References in corpus (9)

• mixup: Beyond Empirical Risk Minimization
• PVT v2: Improved Baselines with Pyramid Vision Transformer
• VMamba: Visual State Space Model
• Focal Self-attention for Local-Global Interactions in Vision Transformers
• A survey on efficient vision transformers: algorithms, techniques, and performance benchmarking
• Multi-Scale VMamba: Hierarchy in Hierarchy Visual State Space Model
• Demystify Mamba in Vision: A Linear Attention Perspective
• EfficientVMamba: Atrous Selective Scan for Light Weight Visual Mamba
• FViT: A Focal Vision Transformer with Gabor Filter