Generative Capacity of Probabilistic Protein Sequence Models

Abstract

Potts models and variational autoencoders (VAEs) have recently gained popularity as generative protein sequence models (GPSMs) to explore fitness landscapes and predict the effect of mutations. Despite encouraging results, quantitative characterization and comparison of GPSM-generated probability distributions is still lacking. It is currently unclear whether GPSMs can faithfully reproduce the complex multi-residue mutation patterns observed in natural sequences arising due to epistasis. We develop a set of sequence statistics to assess the "generative capacity" of three GPSMs of recent interest: the pairwise Potts Hamiltonian, the VAE, and the site-independent model, using natural and synthetic datasets. We show that the generative capacity of the Potts Hamiltonian model is the largest, in that the higher order mutational statistics generated by the model agree with those observed for natural sequences. In contrast, we show that the VAE's generative capacity lies between the pairwise Potts and site-independent models. Importantly, our work measures GPSM generative capacity in terms of higher-order sequence covariation statistics which we have developed, and provides a new framework for evaluating and interpreting GPSM accuracy that emphasizes the role of epistasis.

References in corpus (27)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Array Programming with NumPy
• Auto-Encoding Variational Bayes
• An Introduction to Variational Autoencoders
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Weak pairwise correlations imply strongly correlated network states in a neural population
• Identification of direct residue contacts in protein-protein interaction by message passing
• Quantification of the effect of mutations using a global probability model of natural sequence variation
• Accelerating Antimicrobial Discovery with Controllable Deep Generative Models and Molecular Dynamics
• Network information and connected correlations
• Coevolutionary landscape inference and the context-dependence of mutations in beta-lactamase TEM-1
• A practical tutorial on autoencoders for nonlinear feature fusion: Taxonomy, models, software and guidelines
• Inverse Statistical Physics of Protein Sequences: A Key Issues Review
• Cyclical Annealing Schedule: A Simple Approach to Mitigating KL Vanishing
• How pairwise coevolutionary models capture the collective residue variability in proteins
• Intrinsic dimension of data representations in deep neural networks
• Efficient generative modeling of protein sequences using simple autoregressive models
• Hyperspherical Variational Auto-Encoders
• Diagnosing and Enhancing VAE Models
• Influence of Multiple Sequence Alignment Depth on Potts Statistical Models of Protein Covariation
• Explorations in Homeomorphic Variational Auto-Encoding
• Rediscovering the power of pairwise interactions
• Variational auto-encoding of protein sequences
• Selection of sequence motifs and generative Hopfield-Potts models for protein familiesilies
• The Usual Suspects? Reassessing Blame for VAE Posterior Collapse
• Guided Variational Autoencoder for Disentanglement Learning
• Variational Autoencoders with Riemannian Brownian Motion Priors

Cited by in corpus (3)

• Protein language models trained on multiple sequence alignments learn phylogenetic relationships
• Generative power of a protein language model trained on multiple sequence alignments
• Phylogenetic Corrections and Higher-Order Sequence Statistics in Protein Families: The Potts Model vs MSA Transformer