101 citations · 261 across the 12 of their papers we have counts for
3 papers · 1 filter
Profile Prediction: An Alignment-Based Pre-Training Task for Protein Sequence Models
Pascal Sturmfels, Jesse Vig, Ali Madani +1
For protein sequence datasets, unlabeled data has greatly outpaced labeled data due to the high cost of wet-lab characterization. Recent deep-learning approaches to protein predict…
BERTology Meets Biology: Interpreting Attention in Protein Language Models
Jesse Vig, Ali Madani, Lav R. Varshney +3
Transformer architectures have proven to learn useful representations for protein classification and generation tasks. However, these representations present challenges in interpre…
Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Bias
Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov +6
Common methods for interpreting neural models in natural language processing typically examine either their structure or their behavior, but not both. We propose a methodology grou…