Sparse generative modeling via parameter-reduction of Boltzmann machines: application to protein-sequence families

Abstract

Boltzmann machines (BM) are widely used as generative models. For example, pairwise Potts models (PM), which are instances of the BM class, provide accurate statistical models of families of evolutionarily related protein sequences. Their parameters are the local fields, which describe site-specific patterns of amino-acid conservation, and the two-site couplings, which mirror the coevolution between pairs of sites. This coevolution reflects structural and functional constraints acting on protein sequences during evolution. The most conservative choice to describe the coevolution signal is to include all possible two-site couplings into the PM. This choice, typical of what is known as Direct Coupling Analysis, has been successful for predicting residue contacts in the three-dimensional structure, mutational effects, and in generating new functional sequences. However, the resulting PM suffers from important over-fitting effects: many couplings are small, noisy and hardly interpretable; the PM is close to a critical point, meaning that it is highly sensitive to small parameter perturbations. In this work, we introduce a general parameter-reduction procedure for BMs, via a controlled iterative decimation of the less statistically significant couplings, identified by an information-based criterion that selects either weak or statistically unsupported couplings. For several protein families, our procedure allows one to remove more than $90\%$ of the PM couplings, while preserving the predictive and generative properties of the original dense PM, and the resulting model is far away from criticality, hence more robust to noise.

arXiv comment7 pages, 5 figures, plus Appendix

References in corpus (19)

• Direct-coupling analysis of residue co-evolution captures native contacts across many protein families
• Identification of direct residue contacts in protein-protein interaction by message passing
• Quantification of the effect of mutations using a global probability model of natural sequence variation
• The Bethe lattice spin glass revisited
• Are biological systems poised at criticality?
• Improved contact prediction in proteins: Using pseudolikelihoods to infer Potts models
• Coevolutionary landscape inference and the context-dependence of mutations in beta-lactamase TEM-1
• Inverse statistical problems: from the inverse Ising problem to data science
• Inverse Statistical Physics of Protein Sequences: A Key Issues Review
• Fast pseudolikelihood maximization for direct-coupling analysis of protein structure from many homologous amino-acid sequences
• How pairwise coevolutionary models capture the collective residue variability in proteins
• Improving contact prediction along three dimensions
• Inter-residue, inter-protein and inter-family coevolution: bridging the scales
• Pseudolikelihood Decimation Algorithm Improving the Inference of the Interaction Network in a General Class of Ising Models
• Selection of sequence motifs and generative Hopfield-Potts models for protein familiesilies
• Equilibrium and non-Equilibrium regimes in the learning of Restricted Boltzmann Machines
• Large Pseudo-Counts and $L_2$-Norm Penalties Are Necessary for the Mean-Field Inference of Ising and Potts Models
• Inference of compressed Potts graphical models
• Correlation-Compressed Direct Coupling Analysis

Cited by in corpus (10)

• Efficient generative modeling of protein sequences using simple autoregressive models
• Equilibrium and non-Equilibrium regimes in the learning of Restricted Boltzmann Machines
• Modeling sequence-space exploration and emergence of epistatic signals in protein evolution
• Emergent time scales of epistasis in protein evolution
• Optimal regularizations for data generation with probabilistic graphical models
• Towards Parsimonious Generative Modeling of RNA Families
• Inferring couplings in networks across order-disorder phase transitions
• Unlearning regularization for Boltzmann Machines
• adabmDCA 2.0 -- a flexible but easy-to-use package for Direct Coupling Analysis
• adabmDCA: Adaptive Boltzmann machine learning for biological sequences