Numerical Approximation of Partial Differential Equations by a Variable Projection Method with Artificial Neural Networks
arXiv:2201.09989 · doi:10.1016/j.cma.2022.115284
Abstract
We present a method for solving linear and nonlinear PDEs based on the variable projection (VarPro) framework and artificial neural networks (ANN). For linear PDEs, enforcing the boundary/initial value problem on the collocation points leads to a separable nonlinear least squares problem about the network coefficients. We reformulate this problem by the VarPro approach to eliminate the linear output-layer coefficients, leading to a reduced problem about the hidden-layer coefficients only. The reduced problem is solved first by the nonlinear least squares method to determine the hidden-layer coefficients, and then the output-layer coefficients are computed by the linear least squares method. For nonlinear PDEs, enforcing the boundary/initial value problem on the collocation points leads to a nonlinear least squares problem that is not separable, which precludes the VarPro strategy for such problems. To enable the VarPro approach for nonlinear PDEs, we first linearize the problem with a Newton iteration, using a particular form of linearization. The linearized system is solved by the VarPro framework together with ANNs. Upon convergence of the Newton iteration, the network coefficients provide the representation of the solution field to the original nonlinear problem. We present ample numerical examples with linear and nonlinear PDEs to demonstrate the performance of the method herein. For smooth field solutions, the errors of the current method decrease exponentially as the number of collocation points or the number of output-layer coefficients increases. We compare the current method with the ELM method from a previous work. Under identical conditions and network configurations, the current method exhibits an accuracy significantly superior to the ELM method.
arXiv comment35 pages, 18 figures, 10 tables
References in corpus (20)
- Adam: A Method for Stochastic Optimization
- DeepXDE: A deep learning library for solving differential equations
- DGM: A deep learning algorithm for solving partial differential equations
- An Energy Approach to the Solution of Partial Differential Equations in Computational Mechanics via Machine Learning: Concepts, Implementation and Applications
- Weak Adversarial Networks for High-dimensional Partial Differential Equations
- Local Extreme Learning Machines and Domain Decomposition for Solving Linear and Nonlinear Partial Differential Equations
- A Method for Representing Periodic Functions and Enforcing Exactly Periodic Boundary Conditions with Deep Neural Networks
- Sparse Principal Component Analysis via Variable Projection
- Numerical Approximation of Incompressible Navier-Stokes Equations Based on an Auxiliary Energy Variable
- MgNet: A Unified Framework of Multigrid and Convolutional Neural Network
- Multiphase Flows of N Immiscible Incompressible Fluids: A Reduction-Consistent and Thermodynamically-Consistent Formulation and Associated Algorithm
- Data-driven learning of robust nonlocal physics from high-fidelity synthetic data
- A Roadmap for Discretely Energy-Stable Schemes for Dissipative Systems Based on a Generalized Auxiliary Variable with Guaranteed Positivity
- Estimating Nuisance Parameters in Inverse Problems
- On Computing the Hyperparameter of Extreme Learning Machines: Algorithm and Application to Computational PDEs, and Comparison with Classical and High-Order Finite Elements
- An Unconditionally Energy-Stable Scheme Based on an Implicit Auxiliary Energy Variable for Incompressible Two-Phase Flows with Different Densities Involving Only Precomputable Coefficient Matrices
- A Convective-like Energy-Stable Open Boundary Condition for Simulations of Incompressible Flows
- A Modified Batch Intrinsic Plasticity Method for Pre-training the Random Coefficients of Extreme Learning Machines
- A generalization of variable elimination for separable inverse problems beyond least squares
- Solving PDEs on Unknown Manifolds with Machine Learning