Why are nonlinear fits so challenging?

Abstract

Fitting model parameters to experimental data is a common yet often challenging task, especially if the model contains many parameters. Typically, algorithms get lost in regions of parameter space in which the model is unresponsive to changes in parameters, and one is left to make adjustments by hand. We explain this difficulty by interpreting the fitting process as a generalized interpretation procedure. By considering the manifold of all model predictions in data space, we find that cross sections have a hierarchy of widths and are typically very narrow. Algorithms become stuck as they move near the boundaries. We observe that the model manifold, in addition to being tightly bounded, has low extrinsic curvature, leading to the use of geodesics in the fitting process. We improve the convergence of the Levenberg-Marquardt algorithm by adding the geodesic acceleration to the usual Levenberg-Marquardt step.

4 pages, 4 figures

References in corpus (2)

• Universally Sloppy Parameter Sensitivities in Systems Biology
• The sloppy model universality class and the Vandermonde matrix

Cited by in corpus (47)

• Parameter Space Compression Underlies Emergent Theories and Predictive Models
• The geometry of nonlinear least squares with applications to sloppy models and optimization
• Detecting binary neutron star systems with spin in advanced gravitational-wave detectors
• The limitations of model-based experimental design and parameter estimation in sloppy systems
• Dihedral-angle-corrected registry-dependent interlayer potential for multilayer graphene structures
• Bridging Mechanistic and Phenomenological Models of Complex Biological Systems
• Ephemeral data derived potentials for random structure search
• Simple predictions from multifield inflationary models
• Maximizing the information learned from finite data selects a simple model
• Bayesian Structural Inference for Hidden Processes
• Manifold learning for parameter reduction
• Delineating Parameter Unidentifiabilities in Complex Models
• Cause and Cure of Sloppiness in Ordinary Differential Equation Models
• Avalanche Spatial Structure and Multivariable Scaling Functions; Sizes, Heights, Widths, and Views through Windows
• Stochastic Local Interaction (SLI) Model: Interfacing Machine Learning and Geostatistics
• D${}^3$PO - Denoising, Deconvolving, and Decomposing Photon Observations
• Light Microscopy at Maximal Precision
• Sloppy nuclear energy density functionals: effective model reduction
• The VLA-COSMOS 3 GHz Large Project: Average radio spectral energy distribution of highly star-forming galaxies
• Bayesian, frequentist, and information geometric approaches to parametric uncertainty quantification of classical empirical interatomic potentials
• Chebyshev approximation and the global geometry of sloppy models
• A KIM-compliant potfit for fitting sloppy interatomic potentials: Application to the EDIP model for silicon
• Inferring the spatiotemporal DNA replication program from noisy data
• New Confocal Hyperbola-based Ellipse Fitting with Applications to Estimating Parameters of Mechanical Pipes from Point Clouds
• NY Vir Eclipse timing observations: Two planet solution
• "Sloppy" nuclear energy density functionals (II): Finite nuclei
• Structure Identifiability of an NDS with LFT Parametrized Subsystems
• Fitting in a complex chi^2 landscape using an optimized hypersurface sampling
• Structural Susceptibility and Separation of Time Scales in the van der Pol Oscillator
• Visualizing probabilistic models in Minkowski space with intensive symmetrized Kullback-Leibler embedding
• Online storage ring optimization using dimension-reduction and genetic algorithms
• Fitting a function to time-dependent ensemble averaged data
• Global Structure Identifiability and Reconstructibility of an NDS with Descriptor Subsystems
• Unwinding the model manifold: choosing similarity measures to remove local minima in sloppy dynamical systems
• Data-driven modeling and parameter estimation of Nonlinear systems
• The VLA-COSMOS 3 GHz Large Project: Average radio spectral energy distribution of active galactic nuclei
• Far from Asymptopia
• Beyond Fisher Forecasting for Cosmology
• Stability of the manifold boundary approximation method for reductions of nuclear structure models
• Sparse model from optimal nonuniform embedding of time series
• Discovering sparse control strategies in C. elegans
• Applications of information geometry to spiking neural network behavior
• Parameters inference and model reduction for the Single-Particle Model of Li ion cells
• Extending OpenKIM with an Uncertainty Quantification Toolkit for Molecular Modeling
• Exploration of the Parameter Space in Macroeconomic Agent-Based Models
• On the Parameter Combinations That Matter and on Those That do Not
• Biases in particle localization algorithms