Neural Symbolic Regression that Scales
arXiv:2106.06427
Abstract
Symbolic equations are at the core of scientific discovery. The task of discovering the underlying equation from a set of input-output pairs is called symbolic regression. Traditionally, symbolic regression methods use hand-designed strategies that do not improve with experience. In this paper, we introduce the first symbolic regression method that leverages large scale pre-training. We procedurally generate an unbounded set of equations, and simultaneously pre-train a Transformer to predict the symbolic equation from a corresponding set of input-output-pairs. At test time, we query the model on a new set of points and use its output to guide the search for the equation. We show empirically that this approach can re-discover a set of well-known physical equations, and that it improves over time with more data and compute.
arXiv commentAccepted at the 38th International Conference on Machine Learning (ICML) 2021
References in corpus (9)
- Learning Transferable Visual Models From Natural Language Supervision
- Language Models are Few-Shot Learners
- Reconciling modern machine learning practice and the bias-variance trade-off
- Scaling Laws for Neural Language Models
- Discovering Symbolic Models from Deep Learning with Inductive Biases
- Analysing Mathematical Reasoning Abilities of Neural Models
- AI Feynman 2.0: Pareto-optimal symbolic regression exploiting graph modularity
- Deep Learning for Symbolic Mathematics
- Generative Language Modeling for Automated Theorem Proving