The sloppy model universality class and the Vandermonde matrix

Abstract

In a variety of contexts, physicists study complex, nonlinear models with many unknown or tunable parameters to explain experimental data. We explain why such systems so often are sloppy; the system behavior depends only on a few `stiff' combinations of the parameters and is unchanged as other `sloppy' parameter combinations vary by orders of magnitude. We contrast examples of sloppy models (from systems biology, variational quantum Monte Carlo, and common data fitting) with systems which are not sloppy (multidimensional linear regression, random matrix ensembles). We observe that the eigenvalue spectra for the sensitivity of sloppy models have a striking, characteristic form, with a density of logarithms of eigenvalues which is roughly constant over a large range. We suggest that the common features of sloppy models indicate that they may belong to a common universality class. In particular, we motivate focusing on a Vandermonde ensemble of multiparameter nonlinear models and show in one limit that they exhibit the universal features of sloppy models.

arXiv commentNew content added

References in corpus (4)

• Random-Matrix Theory of Quantum Transport
• Random Matrix Theories in Quantum Physics: Common Concepts
• The statistical mechanics of complex signaling networks : nerve growth factor signaling
• Bayesian Error Estimation in Density Functional Theory

Cited by in corpus (47)

• Universally Sloppy Parameter Sensitivities in Systems Biology
• Parameter Space Compression Underlies Emergent Theories and Predictive Models
• The geometry of nonlinear least squares with applications to sloppy models and optimization
• Sloppiness, robustness, and evolvability in systems biology
• Why are nonlinear fits so challenging?
• Nonmonotonic Aging and Memory Retention in Disordered Mechanical Systems
• The limitations of model-based experimental design and parameter estimation in sloppy systems
• PCA meets RG
• Bridging Mechanistic and Phenomenological Models of Complex Biological Systems
• Geometry of learning neural quantum states
• Extracting falsifiable predictions from sloppy models
• Maximizing the information learned from finite data selects a simple model
• Energy-entropy competition and the effectiveness of stochastic gradient descent in machine learning
• Manifold learning for parameter reduction
• Delineating Parameter Unidentifiabilities in Complex Models
• Cause and Cure of Sloppiness in Ordinary Differential Equation Models
• Sloppy nuclear energy density functionals: effective model reduction
• KLIFF: A framework to develop physics-based and machine learning interatomic potentials
• Bayesian, frequentist, and information geometric approaches to parametric uncertainty quantification of classical empirical interatomic potentials
• Seven Lessons from Manyfield Inflation in Random Potentials
• Thermodynamic curvature for a two-parameter spin model with frustration
• Chebyshev approximation and the global geometry of sloppy models
• A KIM-compliant potfit for fitting sloppy interatomic potentials: Application to the EDIP model for silicon
• Universal properties of concentration sensing in large ligand-receptor networks
• "Sloppy" nuclear energy density functionals (II): Finite nuclei
• Variational inference of fractional Brownian motion with linear computational complexity
• Two-level modeling of quarantine
• Visualizing probabilistic models in Minkowski space with intensive symmetrized Kullback-Leibler embedding
• Fitting in a complex chi^2 landscape using an optimized hypersurface sampling
• Random Vandermonde Matrices-Part II: Applications
• Signal detection in nearly continuous spectra and symmetry breaking
• Inflation as an Information Bottleneck - A strategy for identifying universality classes and making robust predictions
• Optimal experimental design in an EGFR signaling and down-regulation model
• Variational method for estimating the rate of convergence of Markov Chain Monte Carlo algorithms
• Frequency Domain Identifiability and Sloppiness of Descriptor Systems with an LFT Structure
• Unwinding the model manifold: choosing similarity measures to remove local minima in sloppy dynamical systems
• Principles underlying efficient exciton transport unveiled by information-geometric analysis
• Contact-mediated cellular communication supplements positional information to regulate spatial patterning during development
• Dimension reduction for systems with slow relaxation
• Does the Data Induce Capacity Control in Deep Learning?
• Classical and Bayesian error analysis of the relativistic mean-field model for doubly magic nuclei
• Sloppy models can be identifiable
• The limits of statistical significance of Hawkes processes fitted to financial data
• Parameters inference and model reduction for the Single-Particle Model of Li ion cells
• Managing Expectations and Imbalanced Training Data in Reactive Force Field Development: an Application to Water Adsorption on Alumina
• Adaptive mutation of biochemical reaction constants: Fisher's geometrical model without pleiotropy
• Extending OpenKIM with an Uncertainty Quantification Toolkit for Molecular Modeling