Priors for symbolic regression

Abstract

When choosing between competing symbolic models for a data set, a human will naturally prefer the "simpler" expression or the one which more closely resembles equations previously seen in a similar context. This suggests a non-uniform prior on functions, which is, however, rarely considered within a symbolic regression (SR) framework. In this paper we develop methods to incorporate detailed prior information on both functions and their parameters into SR. Our prior on the structure of a function is based on a $n$-gram language model, which is sensitive to the arrangement of operators relative to one another in addition to the frequency of occurrence of each operator. We also develop a formalism based on the Fractional Bayes Factor to treat numerical parameter priors in such a way that models may be fairly compared though the Bayesian evidence, and explicitly compare Bayesian, Minimum Description Length and heuristic methods for model selection. We demonstrate the performance of our priors relative to literature standards on benchmarks and a real-world dataset from the field of cosmology.

arXiv comment8+2 pages, 2 figures. Accepted for The Genetic and Evolutionary Computation Conference (GECCO) 2023 Workshop on Symbolic Regression

References in corpus (10)

• A Comprehensive Measurement of the Local Value of the Hubble Constant with 1 km/s/Mpc Uncertainty from the Hubble Space Telescope and the SH0ES Team
• A Bayesian machine scientist to aid in the solution of challenging scientific problems
• SymbolicGPT: A Generative Transformer Model for Symbolic Regression
• Probabilistic Grammars for Equation Discovery
• Deep Symbolic Regression for Recurrent Sequences
• Transformer-based Planning for Symbolic Regression
• SymFormer: End-to-end symbolic regression using transformer-based architecture
• Automated Learning of Interpretable Models with Quantified Uncertainty
• End-to-end symbolic regression with transformers
• Discovering ordinary differential equations that govern time-series

Cited by in corpus (5)

• syren-halofit: A fast, interpretable, high-precision formula for the $\Lambda$CDM nonlinear matter power spectrum
• syren-new: Precise formulae for the linear and nonlinear matter power spectra with massive neutrinos and dynamical dark energy
• Optimal Inflationary Potentials
• cp3-bench: A tool for benchmarking symbolic regression algorithms tested with cosmology
• Constraining dark matter halo profiles with symbolic regression