A Nonoverlapping Domain Decomposition Method for Extreme Learning Machines: Elliptic Problems

Abstract

Extreme learning machine (ELM) is a methodology for solving partial differential equations (PDEs) using a single hidden layer feed-forward neural network. It presets the weight/bias coefficients in the hidden layer with random values, which remain fixed throughout the computation, and uses a linear least squares method for training the parameters of the output layer of the neural network. It is known to be much faster than Physics informed neural networks. However, classical ELM is still computationally expensive when a high level of representation is desired in the solution as this requires solving a large least squares system. In this paper, we propose a nonoverlapping domain decomposition method (DDM) for ELMs that not only reduces the training time of ELMs, but is also suitable for parallel computation. In numerical analysis, DDMs have been widely studied to reduce the time to obtain finite element solutions for elliptic PDEs through parallel computation. Among these approaches, nonoverlapping DDMs are attracting the most attention. Motivated by these methods, we introduce local neural networks, which are valid only at corresponding subdomains, and an auxiliary variable at the interface. We construct a system on the variable and the parameters of local neural networks. A Schur complement system on the interface can be derived by eliminating the parameters of the output layer. The auxiliary variable is then directly obtained by solving the reduced system after which the parameters for each local neural network are solved in parallel. A method for initializing the hidden layer parameters suitable for high approximation quality in large systems is also proposed. Numerical results that verify the acceleration performance of the proposed method with respect to the number of subdomains are presented.

18 pages, 4 figures, 7 tables

References in corpus (9)

• Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
• Finite Basis Physics-Informed Neural Networks (FBPINNs): a scalable domain decomposition approach for solving differential equations
• Local Extreme Learning Machines and Domain Decomposition for Solving Linear and Nonlinear Partial Differential Equations
• Construction of arbitrary order finite element degree-of-freedom maps on polygonal and polyhedral cell meshes
• Multilevel domain decomposition-based architectures for physics-informed neural networks
• Enhancing training of physics-informed neural networks using domain-decomposition based preconditioning strategies
• Additive Schwarz Methods for Convex Optimization as Gradient Methods
• Two-level Group Convolution
• Parareal Neural Networks Emulating a Parallel-in-time Algorithm

Cited by in corpus (1)

• A Neumann-Neumann Acceleration with Coarse Space for Domain Decomposition of Extreme Learning Machines