Feature Engineering with Regularity Structures
arXiv:2108.05879 · doi:10.1007/s10915-023-02401-4
Abstract
We investigate the use of models from the theory of regularity structures as features in machine learning tasks. A model is a polynomial function of a space-time signal designed to well-approximate solutions to partial differential equations (PDEs), even in low regularity regimes. Models can be seen as natural multi-dimensional generalisations of signatures of paths; our work therefore aims to extend the recent use of signatures in data science beyond the context of time-ordered data. We provide a flexible definition of a model feature vector associated to a space-time signal, along with two algorithms which illustrate ways in which these features can be combined with linear regression. We apply these algorithms in several numerical experiments designed to learn solutions to PDEs with a given forcing and boundary data. Our experiments include semi-linear parabolic and wave equations with forcing, and Burgers' equation with no forcing. We find an advantage in favour of our algorithms when compared to several alternative methods. Additionally, in the experiment with Burgers' equation, we find non-trivial predictive power when noise is added to the observations.
arXiv comment33 pages, 7 figures, 7 tables. Improved presentation of model feature vector (Section 2) and experiments (Section 3). Added new experiment in 2D spatial domain (Section 3.1.2). To appear in Journal of Scientific Computing
References in corpus (16)
- Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
- Machine Learning for Fluid Mechanics
- DGM: A deep learning algorithm for solving partial differential equations
- Solving high-dimensional partial differential equations using deep learning
- A theory of regularity structures
- Learning data driven discretizations for partial differential equations
- Uniqueness for the signature of a path of bounded variation and the reduced path group
- Deep Nitsche Method: Deep Ritz Method with Essential Boundary Conditions
- Sparse arrays of signatures for online character recognition
- A signature-based machine learning model for bipolar disorder and borderline personality disorder
- Persistence paths and signature features in topological data analysis
- Characteristic functions of measures on geometric rough paths
- Neural Networks Trained to Solve Differential Equations Learn General Representations
- Signature moments to characterize laws of stochastic processes
- An augmented Lagrangian deep learning method for variational problems with essential boundary conditions
- Parameter estimation for rough differential equations