ProtVec: A Continuous Distributed Representation of Biological Sequences

Abstract

We introduce a new representation and feature extraction method for biological sequences. Named bio-vectors (BioVec) to refer to biological sequences in general with protein-vectors (ProtVec) for proteins (amino-acid sequences) and gene-vectors (GeneVec) for gene sequences, this representation can be widely used in applications of deep learning in proteomics and genomics. In the present paper, we focus on protein-vectors that can be utilized in a wide array of bioinformatics investigations such as family classification, protein visualization, structure prediction, disordered protein identification, and protein-protein interaction prediction. In this method, we adopt artificial neural network approaches and represent a protein sequence with a single dense n-dimensional vector. To evaluate this method, we apply it in classification of 324,018 protein sequences obtained from Swiss-Prot belonging to 7,027 protein families, where an average family classification accuracy of 93%+-0.06% is obtained, outperforming existing family classification methods. In addition, we use ProtVec representation to predict disordered proteins from structured proteins. Two databases of disordered sequences are used: the DisProt database as well as a database featuring the disordered regions of nucleoporins rich with phenylalanine-glycine repeats (FG-Nups). Using support vector machine classifiers, FG-Nup sequences are distinguished from structured protein sequences found in Protein Data Bank (PDB) with a 99.8% accuracy, and unstructured DisProt sequences are differentiated from structured DisProt sequences with 100.0% accuracy. These results indicate that by only providing sequence data for various proteins into this model, accurate information about protein structure can be determined.

References in corpus (3)

• Distributed Representations of Words and Phrases and their Compositionality
• Natural Language Processing (almost) from Scratch
• word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method

Cited by in corpus (37)

• DeepER -- Deep Entity Resolution
• Controllable Protein Design with Language Models
• Anomaly Detection using One-Class Neural Networks
• Exploring Chemical Space using Natural Language Processing Methodologies for Drug Discovery
• Design of metalloproteins and novel protein folds using variational autoencoders
• Protein Secondary Structure Prediction Using Cascaded Convolutional and Recurrent Neural Networks
• Linguistically inspired roadmap for building biologically reliable protein language models
• Distributed Representations for Biological Sequence Analysis
• A novel methodology on distributed representations of proteins using their interacting ligands
• Drug-Target Interaction/Affinity Prediction: Deep Learning Models and Advances Review
• DeepGS: Deep Representation Learning of Graphs and Sequences for Drug-Target Binding Affinity Prediction
• Mixed Dimension Embeddings with Application to Memory-Efficient Recommendation Systems
• Advancing bioinformatics with large language models: components, applications and perspectives
• Ontology Embedding: A Survey of Methods, Applications and Resources
• Pre-Training of Deep Bidirectional Protein Sequence Representations with Structural Information
• ChemBoost: A chemical language based approach for protein-ligand binding affinity prediction
• Modeling Protein Using Large-scale Pretrain Language Model
• Deep Learning in Bioinformatics
• Intrinsic-Extrinsic Convolution and Pooling for Learning on 3D Protein Structures
• Deep Learning in Protein Structural Modeling and Design
• Review of Natural Language Processing in Pharmacology
• MC-NN: An End-to-End Multi-Channel Neural Network Approach for Predicting Influenza A Virus Hosts and Antigenic Types
• Numeric Lyndon-based feature embedding of sequencing reads for machine learning approaches
• MUFold-SS: Protein Secondary Structure Prediction Using Deep Inception-Inside-Inception Networks
• Virus genome sequence classification using features based on nucleotides, words and compression
• Learning from Data-Rich Problems: A Case Study on Genetic Variant Calling
• A Robust and Precise ConvNet for small non-coding RNA classification (RPC-snRC)
• Comprehensive Evaluation of Deep Learning Architectures for Prediction of DNA/RNA Sequence Binding Specificities
• A New Approach for Scalable Analysis of Microbial Communities
• A Categorical Compositional Distributional Modelling for the Language of Life
• Nonsymbolic Text Representation
• Machine Learning for Protein Function
• Improving Chinese Segmentation-free Word Embedding With Unsupervised Association Measure
• Towards the Latent Transcriptome
• A comprehensive empirical analysis on cross-domain semantic enrichment for detection of depressive language
• Multi-view Banded Spectral Clustering with Application to ICD9 Clustering
• HLA class I binding prediction via convolutional neural networks