Do Transformer Models Show Similar Attention Patterns to Task-Specific Human Gaze?

Stephanie Brandl, Oliver Eberle, Jonas Pilot +1

Learned self-attention functions in state-of-the-art NLP models often correlate with human attention. We investigate whether self-attention in large-scale pre-trained language mode…