Structure-Aware Masking for Protein Representation Learning

Thomas Walton, Ayan Goel, Amirali Aghazadeh

Masked language modeling (MLM) is the standard objective for training protein language models, typically implemented by randomly masking individual residues at a fixed rate (e.g.,…