Generative Capacity of Probabilistic Protein Sequence Models

Abstract

Potts models and variational autoencoders (VAEs) have recently gained popularity as generative protein sequence models (GPSMs) to explore fitness landscapes and predict the effect of mutations. Despite encouraging results, quantitative characterization and comparison of GPSM-generated probability distributions is still lacking. It is currently unclear whether GPSMs can faithfully reproduce the complex multi-residue mutation patterns observed in natural sequences arising due to epistasis. We develop a set of sequence statistics to assess the "generative capacity" of three GPSMs of recent interest: the pairwise Potts Hamiltonian, the VAE, and the site-independent model, using natural and synthetic datasets. We show that the generative capacity of the Potts Hamiltonian model is the largest, in that the higher order mutational statistics generated by the model agree with those observed for natural sequences. In contrast, we show that the VAE's generative capacity lies between the pairwise Potts and site-independent models. Importantly, our work measures GPSM generative capacity in terms of higher-order sequence covariation statistics which we have developed, and provides a new framework for evaluating and interpreting GPSM accuracy that emphasizes the role of epistasis.

References in corpus (13)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Array Programming with NumPy
• Identification of direct residue contacts in protein-protein interaction by message passing
• Accelerating Antimicrobial Discovery with Controllable Deep Generative Models and Molecular Dynamics
• A practical tutorial on autoencoders for nonlinear feature fusion: Taxonomy, models, software and guidelines
• Cyclical Annealing Schedule: A Simple Approach to Mitigating KL Vanishing
• Efficient generative modeling of protein sequences using simple autoregressive models
• Influence of Multiple Sequence Alignment Depth on Potts Statistical Models of Protein Covariation
• Rediscovering the power of pairwise interactions
• Variational auto-encoding of protein sequences
• The Usual Suspects? Reassessing Blame for VAE Posterior Collapse
• Guided Variational Autoencoder for Disentanglement Learning
• Variational Autoencoders with Riemannian Brownian Motion Priors

Cited by in corpus (3)

• Protein language models trained on multiple sequence alignments learn phylogenetic relationships
• Generative power of a protein language model trained on multiple sequence alignments
• Phylogenetic Corrections and Higher-Order Sequence Statistics in Protein Families: The Potts Model vs MSA Transformer