Efficient Generator of Mathematical Expressions for Symbolic Regression

Abstract

We propose an approach to symbolic regression based on a novel variational autoencoder for generating hierarchical structures, HVAE. It combines simple atomic units with shared weights to recursively encode and decode the individual nodes in the hierarchy. Encoding is performed bottom-up and decoding top-down. We empirically show that HVAE can be trained efficiently with small corpora of mathematical expressions and can accurately encode expressions into a smooth low-dimensional latent space. The latter can be efficiently explored with various optimization methods to address the task of symbolic regression. Indeed, random search through the latent space of HVAE performs better than random search through expressions generated by manually crafted probabilistic grammars for mathematical expressions. Finally, EDHiE system for symbolic regression, which applies an evolutionary algorithm to the latent space of HVAE, reconstructs equations from a standard symbolic regression benchmark better than a state-of-the-art system based on a similar combination of deep learning and evolutionary algorithms.ž

arXiv comment35 pages, 11 tables, 7 multi-part figures, Machine learning (Springer) and journal track of ECML/PKDD 2023

References in corpus (12)

• Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network
• Automatic chemical design using a data-driven continuous representation of molecules
• pymoo: Multi-objective Optimization in Python
• On the Properties of Neural Machine Translation: Encoder-Decoder Approaches
• Syntax-Directed Variational Autoencoder for Structured Data
• Interpretable Machine Learning for Science with PySR and SymbolicRegression.jl
• A Bayesian machine scientist to aid in the solution of challenging scientific problems
• Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients
• Learning Continuous Semantic Representations of Symbolic Expressions
• Probabilistic Grammars for Equation Discovery
• Natural Language Generation with Neural Variational Models
• Learning a Generative Model for Validity in Complex Discrete Structures

Cited by in corpus (2)

• SPINEX_ Symbolic Regression: Similarity-based Symbolic Regression with Explainable Neighbors Exploration
• Approximating the universal thermal climate index using sparse regression with orthogonal polynomials