Inhibited Self-Attention: Sharpening Focus in Vision Transformers

Peter R. D. van der Wal, Nicola Strisciuglio, George Azzopardi

The paper proposes Inhibited Self-Attention (ISA), a modification to Vision Transformers that incorporates negative attention scores to suppress background features, leading to sha…