Shape-constrained Symbolic Regression -- Improving Extrapolation with Prior Knowledge

Abstract

We investigate the addition of constraints on the function image and its derivatives for the incorporation of prior knowledge in symbolic regression. The approach is called shape-constrained symbolic regression and allows us to enforce e.g. monotonicity of the function over selected inputs. The aim is to find models which conform to expected behaviour and which have improved extrapolation capabilities. We demonstrate the feasibility of the idea and propose and compare two evolutionary algorithms for shape-constrained symbolic regression: i) an extension of tree-based genetic programming which discards infeasible solutions in the selection step, and ii) a two population evolutionary algorithm that separates the feasible from the infeasible solutions. In both algorithms we use interval arithmetic to approximate bounds for models and their partial derivatives. The algorithms are tested on a set of 19 synthetic and four real-world regression problems. Both algorithms are able to identify models which conform to shape constraints which is not the case for the unmodified symbolic regression algorithms. However, the predictive accuracy of models with constraints is worse on the training set and the test set. Shape-constrained polynomial regression produces the best results for the test set but also significantly larger models.

References in corpus (11)

• Gene Expression Programming: a New Adaptive Algorithm for Solving Problems
• Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data
• Integration of Neural Network-Based Symbolic Regression in Deep Learning for Scientific Discovery
• Where are we now? A large benchmark study of recent symbolic regression methods
• A Greedy Search Tree Heuristic for Symbolic Regression
• Monotonic Calibrated Interpolated Look-Up Tables
• Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients
• Learning Equations for Extrapolation and Control
• Optimization over Nonnegative and Convex Polynomials With and Without Semidefinite Programming
• Neural-Guided Symbolic Regression with Asymptotic Constraints
• Shape-Constrained Regression using Sum of Squares Polynomials

Cited by in corpus (10)

• A computational framework for physics-informed symbolic regression with straightforward integration of domain knowledge
• Symbolic Regression as Feature Engineering Method for Machine and Deep Learning Regression Tasks
• Toward Physically Plausible Data-Driven Models: A Novel Neural Network Approach to Symbolic Regression
• Alleviating Overfitting in Transformation-Interaction-Rational Symbolic Regression with Multi-Objective Optimization
• Improving Genetic Programming for Symbolic Regression with Equality Graphs
• Using Shape Constraints for Improving Symbolic Regression Models
• Comparing Shape-Constrained Regression Algorithms for Data Validation
• rEGGression: an Interactive and Agnostic Tool for the Exploration of Symbolic Regression Models
• Shape-constrained Symbolic Regression with NSGA-III
• Can Synthetic Data Improve Symbolic Regression Extrapolation Performance?