cp3-bench: A tool for benchmarking symbolic regression algorithms tested with cosmology

Abstract

We introduce cp3-bench, a tool for comparing symbolic regression algorithms which we make publicly available at https://github.com/CP3-Origins/cp3-bench. Currently, cp3-bench includes 12 symbolic regression algorithms which can be automatically installed as part of cp3-bench. The philosophy behind cp3-bench is that it should be as user-friendly as possible, available in a ready-to-use format, and allow for easy additions of new algorithms and datasets. Our hope is that users of symbolic regression algorithms can use cp3-bench to easily install and compare symbolic regression algorithms to better decide which algorithms to use for their specific tasks at hand. To introduce and motivate the use of cp3-bench we present a benchmark of 12 symbolic regression algorithms applied to 28 datasets representing six different astrophysical setups. Overall, we find that most of the benched algorithms do poorly in the benchmark and suggest possible ways to proceed with developing algorithms that will be better at identifying ground truth expressions for cosmological datasets. Our demonstration benchmark studies the significance of dimensionality of the feature space and precision of datasets. We find both to be highly important for symbolic regression tasks to be successful. On the other hand, we find no indication that inter-dependence of features in datasets is important, i.e. it is not in general a hindrance for symbolic regression algorithms if datasets e.g. contain both $z$ and $H(z)$ as features. Lastly, we find no indication that performance of algorithms on standardized datasets are good indicators of performance on particular astrophysical datasets. This suggests that it is not necessarily prudent to choose symbolic regression algorithms based on their performance on standardized data. A more robust approach is to consider a variety of algorithms, chosen based on the particular task at hand.

45 pages, 26 tables, 2 figures. v2: 72 pages. Extended benchmark to 12 algorithms and 28 datasets + clarified text. Matches version accepted for publication in JCAP

References in corpus (45)

• Detection of the Baryon Acoustic Peak in the Large-Scale Correlation Function of SDSS Luminous Red Galaxies
• The clustering of galaxies in the completed SDSS-III Baryon Oscillation Spectroscopic Survey: cosmological analysis of the DR12 galaxy sample
• The 6dF Galaxy Survey: Baryon Acoustic Oscillations and the Local Hubble Constant
• Dark Energy and the Accelerating Universe
• The Clustering of the SDSS DR7 Main Galaxy Sample I: A 4 per cent Distance Measure at z=0.15
• Constraining Cosmological Parameters Based on Relative Galaxy Ages
• The Core-Cusp Problem
• Stellar Population Diagnostics of Elliptical Galaxy Formation
• On average properties of inhomogeneous fluids in general relativity II: perfect fluid cosmologies
• Setting the Stage for Cosmic Chronometers. II. Impact of Stellar Population Synthesis Models Systematics and Full Covariance Matrix
• Backreaction in late-time cosmology
• Does the growth of structure affect our dynamical models of the universe? The averaging, backreaction and fitting problems in cosmology
• Dark energy from backreaction
• Interpretable Machine Learning for Science with PySR and SymbolicRegression.jl
• The universe seen at different scales
• Accelerated expansion from structure formation
• Contemporary Symbolic Regression Methods and their Relative Performance
• Light propagation in statistically homogeneous and isotropic dust universes
• Setting the Stage for Cosmic Chronometers. I. Assessing the Impact of Young Stellar Populations on Hubble Parameter Measurements
• Deep symbolic regression for physics guided by units constraints: toward the automated discovery of physical laws
• Light propagation in statistically homogeneous and isotropic universes with general matter content
• Inhomogeneity effects in Cosmology
• AI Feynman 2.0: Pareto-optimal symbolic regression exploiting graph modularity
• Exhaustive Symbolic Regression
• On average properties of inhomogeneous fluids in general relativity III: general fluid cosmologies
• Another look at redshift drift and the backreaction conjecture
• Precision Machine Learning
• Observations in statistically homogeneous, locally inhomogeneous cosmological toy-models without FLRW backgrounds
• Local and non-local measures of acceleration in cosmology
• Searching for signals of inhomogeneity using multiple probes of the cosmic expansion rate H(z)
• Accelerated expansion without dark energy
• Cosmological acceleration from structure formation
• Redshift drift as a model independent probe of dark energy
• On average properties of inhomogeneous cosmologies
• syren-halofit: A fast, interpretable, high-precision formula for the $\Lambda$CDM nonlinear matter power spectrum
• A precise symbolic emulator of the linear matter power spectrum
• Cosmic backreaction and the mean redshift drift from symbolic regression
• Machine learning cosmic backreaction and its effects on observations
• Priors for symbolic regression
• Dimensionless machine learning: Imposing exact units equivariance
• A Complete Analytic Gravitational Wave Model for Undergraduates
• Dark matter halo mass functions and density profiles from mass and energy cascade
• Investigating Dark Matter and MOND Models with Galactic Rotation Curve Data
• A robust two-parameter description of the stellar profile of elliptical galaxies
• Dimensionally Constrained Symbolic Regression

Cited by in corpus (3)

• Call for Action: towards the next generation of symbolic regression benchmark
• Machine Learning-Based Analytical Expressions for Gray-Body Factors and Application to Primordial Black Holes
• Conditional variational autoencoders for cosmological model discrimination and anomaly detection in cosmic microwave background power spectra