Contributions of Transformer Attention Heads in Multi- and Cross-lingual Tasks

Weicheng Ma, Kai Zhang, Renze Lou +2

This paper studies the relative importance of attention heads in Transformer-based models to aid their interpretability in cross-lingual and multi-lingual tasks. Prior research has…