RandONet: Shallow-Networks with Random Projections for learning linear and nonlinear operators

Abstract

Deep Operator Networks (DeepOnets) have revolutionized the domain of scientific machine learning for the solution of the inverse problem for dynamical systems. However, their implementation necessitates optimizing a high-dimensional space of parameters and hyperparameters. This fact, along with the requirement of substantial computational resources, poses a barrier to achieving high numerical accuracy. Here, inpsired by DeepONets and to address the above challenges, we present Random Projection-based Operator Networks (RandONets): shallow networks with random projections that learn linear and nonlinear operators. The implementation of RandONets involves: (a) incorporating random bases, thus enabling the use of shallow neural networks with a single hidden layer, where the only unknowns are the output weights of the network's weighted inner product; this reduces dramatically the dimensionality of the parameter space; and, based on this, (b) using established least-squares solvers (e.g., Tikhonov regularization and preconditioned QR decomposition) that offer superior numerical approximation properties compared to other optimization techniques used in deep-learning. In this work, we prove the universal approximation accuracy of RandONets for approximating nonlinear operators and demonstrate their efficiency in approximating linear nonlinear evolution operators (right-hand-sides (RHS)) with a focus on PDEs. We show, that for this particular task, RandONets outperform, both in terms of numerical approximation accuracy and computational cost, the ``vanilla" DeepOnets.

References in corpus (14)

• DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators
• Solving high-dimensional partial differential equations using deep learning
• A physics-informed variational DeepONet for predicting the crack path in brittle materials
• Local Extreme Learning Machines and Domain Decomposition for Solving Linear and Nonlinear Partial Differential Equations
• Approximation with Random Bases: Pro et Contra
• Coarse-scale PDEs from fine-scale observations via machine learning
• SVD Perspectives for Augmenting DeepONet Flexibility and Interpretability
• Constructing coarse-scale bifurcation diagrams from spatio-temporal observations of microscopic simulations: A parsimonious machine learning approach
• Time Series Forecasting Using Manifold Learning
• Learning the temporal evolution of multivariate densities via normalizing flows
• Data-driven modelling of brain activity using neural networks, Diffusion Maps, and the Koopman operator
• Nonlinear Discrete-Time Observers with Physics-Informed Neural Networks
• Slow Invariant Manifolds of Singularly Perturbed Systems via Physics-Informed Machine Learning
• Discrete-Time Nonlinear Feedback Linearization via Physics-Informed Machine Learning

Cited by in corpus (4)

• GoRINNs: Godunov-Riemann Informed Neural Networks for Learning Hyperbolic Conservation Laws
• Evolutionary Optimization of Physics-Informed Neural Networks: Evo-PINN Frontiers and Opportunities
• Are Two Hidden Layers Still Enough for the Physics-Informed Neural Networks?
• Fredholm Neural Networks