Learning protein sequence embeddings using information from structure

Abstract

Inferring the structural properties of a protein from its amino acid sequence is a challenging yet important problem in biology. Structures are not known for the vast majority of protein sequences, but structure is critical for understanding function. Existing approaches for detecting structural similarity between proteins from sequence are unable to recognize and exploit structural patterns when sequences have diverged too far, limiting our ability to transfer knowledge between structurally related proteins. We newly approach this problem through the lens of representation learning. We introduce a framework that maps any protein sequence to a sequence of vector embeddings --- one per amino acid position --- that encode structural information. We train bidirectional long short-term memory (LSTM) models on protein sequences with a two-part feedback mechanism that incorporates information from (i) global structural similarity between proteins and (ii) pairwise residue contact maps for individual proteins. To enable learning from structural similarity information, we define a novel similarity measure between arbitrary-length sequences of vector embeddings based on a soft symmetric alignment (SSA) between them. Our method is able to learn useful position-specific embeddings despite lacking direct observations of position-level correspondence between sequences. We show empirically that our multi-task framework outperforms other sequence-based methods and even a top-performing structure-based alignment method when predicting structural similarity, our goal. Finally, we demonstrate that our learned embeddings can be transferred to other protein sequence problems, improving the state-of-the-art in transmembrane domain prediction.

17 pages, 3 figures, 8 tables, proceedings of ICLR 2019

Cited by in corpus (13)

• Machine learning modeling of family wide enzyme-substrate specificity screens
• Evaluating Protein Transfer Learning with TAPE
• ProtTrans: Towards Cracking the Language of Life's Code Through Self-Supervised Deep Learning and High Performance Computing
• CogMol: Target-Specific and Selective Drug Design for COVID-19 Using Deep Generative Models
• Pre-trained Language Models in Biomedical Domain: A Systematic Survey
• Is Transfer Learning Necessary for Protein Landscape Prediction?
• Pre-Training of Deep Bidirectional Protein Sequence Representations with Structural Information
• Intrinsic-Extrinsic Convolution and Pooling for Learning on 3D Protein Structures
• Profile Prediction: An Alignment-Based Pre-Training Task for Protein Sequence Models
• Encoding protein dynamic information in graph representation for functional residue identification
• Energy-based models for atomic-resolution protein conformations
• Pre-training Co-evolutionary Protein Representation via A Pairwise Masked Language Model
• Fixed-Length Protein Embeddings using Contextual Lenses