Revisiting Transformers with Insights from Image Filtering and Boosting

Laziz U. Abdullaev, Maksim Tkachenko, Tan M. Nguyen

The self-attention mechanism, a cornerstone of Transformer-based state-of-the-art deep learning architectures, is largely heuristic-driven and fundamentally challenging to interpre…