Your Attention Matters: to Improve Model Robustness to Noise and Spurious Correlations

Camilo Tamayo-Rousseau, Yunjia Zhao, Yiqun Zhang +1

Self-attention mechanisms are foundational to Transformer architectures, supporting their impressive success in a wide range of tasks. While there are many self-attention variants,…