Long-context Protein Language Modeling Using Bidirectional Mamba with Shared Projection Layers

Yingheng Wang, Zichen Wang, Gil Sadeh +4

Self-supervised training of language models (LMs) has seen great success for protein sequences in learning meaningful representations and for generative drug design. Most protein L…