Local Extreme Learning Machines and Domain Decomposition for Solving Linear and Nonlinear Partial Differential Equations

Abstract

We present a neural network-based method for solving linear and nonlinear partial differential equations, by combining the ideas of extreme learning machines (ELM), domain decomposition and local neural networks. The field solution on each sub-domain is represented by a local feed-forward neural network, and $C^k$ continuity is imposed on the sub-domain boundaries. Each local neural network consists of a small number of hidden layers, while its last hidden layer can be wide. The weight/bias coefficients in all hidden layers of the local neural networks are pre-set to random values and are fixed, and only the weight coefficients in the output layers are training parameters. The overall neural network is trained by a linear or nonlinear least squares computation, not by the back-propagation type algorithms. We introduce a block time-marching scheme together with the presented method for long-time dynamic simulations. The current method exhibits a clear sense of convergence with respect to the degrees of freedom in the neural network. Its numerical errors typically decrease exponentially or nearly exponentially as the number of degrees of freedom increases. Extensive numerical experiments have been performed to demonstrate the computational performance of the presented method. We compare the current method with the deep Galerkin method (DGM) and the physics-informed neural network (PINN) in terms of the accuracy and computational cost. The current method exhibits a clear superiority, with its numerical errors and network training time considerably smaller (typically by orders of magnitude) than those of DGM and PINN. We also compare the current method with the classical finite element method (FEM). The computational performance of the current method is on par with, and oftentimes exceeds, the FEM performance.

62 pages, 48 figures, 12 tables

References in corpus (12)

• Adam: A Method for Stochastic Optimization
• Artificial Neural Networks for Solving Ordinary and Partial Differential Equations
• DGM: A deep learning algorithm for solving partial differential equations
• An Energy Approach to the Solution of Partial Differential Equations in Computational Mechanics via Machine Learning: Concepts, Implementation and Applications
• Weak Adversarial Networks for High-dimensional Partial Differential Equations
• A Method for Representing Periodic Functions and Enforcing Exactly Periodic Boundary Conditions with Deep Neural Networks
• D3M: A deep domain decomposition method for partial differential equations
• MgNet: A Unified Framework of Multigrid and Convolutional Neural Network
• The Finite Neuron Method and Convergence Analysis
• Multiphase Flows of N Immiscible Incompressible Fluids: A Reduction-Consistent and Thermodynamically-Consistent Formulation and Associated Algorithm
• Efficient Deep Learning Techniques for Multiphase Flow Simulation in Heterogeneous Porous Media
• A pressure correction scheme for generalized form of energy-stable open boundary conditions for incompressible flows

Cited by in corpus (30)

• Finite Basis Physics-Informed Neural Networks (FBPINNs): a scalable domain decomposition approach for solving differential equations
• Deep Learning in Deterministic Computational Mechanics
• Augmented Physics-Informed Neural Networks (APINNs): A gating network-based soft domain decomposition methodology
• Weak baselines and reporting biases lead to overoptimism in machine learning for fluid-related partial differential equations
• Multilevel domain decomposition-based architectures for physics-informed neural networks
• Physics-informed radial basis network (PIRBN): A local approximating neural network for solving nonlinear PDEs
• On Computing the Hyperparameter of Extreme Learning Machines: Algorithm and Application to Computational PDEs, and Comparison with Classical and High-Order Finite Elements
• Bayesian Physics-Informed Extreme Learning Machine for Forward and Inverse PDE Problems with Noisy Data
• Adaptive deep density approximation for Fokker-Planck equations
• Constructing coarse-scale bifurcation diagrams from spatio-temporal observations of microscopic simulations: A parsimonious machine learning approach
• A Modified Batch Intrinsic Plasticity Method for Pre-training the Random Coefficients of Extreme Learning Machines
• Parsimonious Physics-Informed Random Projection Neural Networks for Initial-Value Problems of ODEs and index-1 DAEs
• A Method for Computing Inverse Parametric PDE Problems with Random-Weight Neural Networks
• CEENs: Causality-enforced evolutional networks for solving time-dependent partial differential equations
• Numerical Approximation of Partial Differential Equations by a Variable Projection Method with Artificial Neural Networks
• Data-driven Control of Agent-based Models: an Equation/Variable-free Machine Learning Approach
• RandONet: Shallow-Networks with Random Projections for learning linear and nonlinear operators
• Neural Networks to solve Partial Differential Equations: a Comparison with Finite Elements
• LSA-PINN: Linear Boundary Connectivity Loss for Solving PDEs on Complex Geometry
• Evolutionary Optimization of Physics-Informed Neural Networks: Evo-PINN Frontiers and Opportunities
• Numerical Solution and Bifurcation Analysis of Nonlinear Partial Differential Equations with Extreme Learning Machines
• A Nonoverlapping Domain Decomposition Method for Extreme Learning Machines: Elliptic Problems
• Fredholm Neural Networks
• Eig-PIELM: A Mesh-Free Approach for Efficient Eigen-Analysis with Physics-Informed Extreme Learning Machines
• A Neumann-Neumann Acceleration with Coarse Space for Domain Decomposition of Extreme Learning Machines
• Adaptive Learning on the Grids for Elliptic Hemivariational Inequalities
• Matched Asymptotic Expansions-Based Transferable Neural Networks for Singular Perturbation Problems
• Numerical Solution of Stiff ODEs with Physics-Informed RPNNs
• Adaptive feature capture method for solving partial differential equations with near singular solutions
• A Novel Fourier Feature Network for Solving Partial Differential Equations