Influence Patterns for Explaining Information Flow in BERT

Kaiji Lu, Zifan Wang, Piotr Mardziel +1

While attention is all you need may be proving true, we do not know why: attention-based transformer models such as BERT are superior but how information flows from input tokens to…