BERTology Meets Biology: Interpreting Attention in Protein Language Models

Jesse Vig, Ali Madani, Lav R. Varshney +3

Transformer architectures have proven to learn useful representations for protein classification and generation tasks. However, these representations present challenges in interpre…