Operator Learning Using Random Features: A Tool for Scientific Computing
arXiv:2408.06526 · doi:10.1137/24M1648703
Abstract
Supervised operator learning centers on the use of training data, in the form of input-output pairs, to estimate maps between infinite-dimensional spaces. It is emerging as a powerful tool to complement traditional scientific computing, which may often be framed in terms of operators mapping between spaces of functions. Building on the classical random features methodology for scalar regression, this paper introduces the function-valued random features method. This leads to a supervised operator learning architecture that is practical for nonlinear problems yet is structured enough to facilitate efficient training through the optimization of a convex, quadratic cost. Due to the quadratic structure, the trained model is equipped with convergence guarantees and error and complexity bounds, properties that are not readily available for most other operator learning architectures. At its core, the proposed approach builds a linear combination of random operators. This turns out to be a low-rank approximation of an operator-valued kernel ridge regression algorithm, and hence the method also has strong connections to Gaussian process regression. The paper designs function-valued random features that are tailored to the structure of two nonlinear operator learning benchmark problems arising from parametric partial differential equations. Numerical results demonstrate the scalability, discretization invariance, and transferability of the function-valued random features method.
arXiv comment36 pages, 1 table, 9 figures. SIGEST version of SIAM J. Sci. Comput. Vol. 43 No. 5 (2021) pp. A3212-A3243, hence text overlap with arXiv:2005.10224
References in corpus (25)
- DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators
- DGM: A deep learning algorithm for solving partial differential equations
- Reconciling modern machine learning practice and the bias-variance trade-off
- Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification
- Learning data driven discretizations for partial differential equations
- Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification
- Stable Architectures for Deep Neural Networks
- Lift & Learn: Physics-informed machine learning for large-scale nonlinear dynamical systems
- Solving parametric PDE problems with artificial neural networks
- Data-Driven Deep Learning of Partial Differential Equations in Modal Space
- Functional Multi-Layer Perceptron: a Nonlinear Tool for Functional Data Analysis
- Solving Electrical Impedance Tomography with Deep Learning
- Operator inference for non-intrusive model reduction with quadratic manifolds
- A physics-informed operator regression framework for extracting data-driven continuum models
- A Mean-Field Optimal Control Formulation of Deep Learning
- Data driven approximation of parametrized PDEs by Reduced Basis and Neural Networks
- Machine Learning from a Continuous Viewpoint
- Non-intrusive model reduction of large-scale, nonlinear dynamical systems using deep learning
- Learning elliptic partial differential equations with randomized linear algebra
- Variational training of neural network approximations of solution maps for physical models
- Generalization Bounds for Sparse Random Feature Expansions
- Meta-learning Pseudo-differential Operators with Deep Neural Networks
- Convergence Rates for Learning Linear Operators from Noisy Data
- Elliptic PDE learning is provably data-efficient
- Data-Driven Forward Discretizations for Bayesian Inversion