Incorporating Residual and Normalization Layers into Analysis of Masked Language Models

Goro Kobayashi, Tatsuki Kuribayashi, Sho Yokoi +1

Transformer architecture has become ubiquitous in the natural language processing field. To interpret the Transformer-based models, their attention patterns have been extensively a…