most citedHow Much Does Attention Actually Attend? Questioning the Importance of Attention in Pretrained Transformers

2 citations · 2 across the 1 of their papers we have counts for

collaborators

1 paper

cs.CL2022★ 2 cited

How Much Does Attention Actually Attend? Questioning the Importance of Attention in Pretrained Transformers

Michael Hassid, Hao Peng, Daniel Rotem +4

The attention mechanism is considered the backbone of the widely-used Transformer architecture. It contextualizes the input by computing input-specific attention matrices. We find…