Distributed Machine Learning for Computational Engineering using MPI

Abstract

We propose a framework for training neural networks that are coupled with partial differential equations (PDEs) in a parallel computing environment. Unlike most distributed computing frameworks for deep neural networks, our focus is to parallelize both numerical solvers and deep neural networks in forward and adjoint computations. Our parallel computing model views data communication as a node in the computational graph for numerical simulations. The advantage of our model is that data communication and computing are cleanly separated and thus provide better flexibility, modularity, and testability. We demonstrate using various large-scale problems that we can achieve substantial acceleration by using parallel solvers for PDEs in training deep neural networks that are coupled with PDEs.

24 pages, 25 figures

References in corpus (8)

• PPINN: Parareal Physics-Informed Neural Network for time-dependent PDEs
• Learning Constitutive Relations from Indirect Observations Using Deep Neural Networks
• Learning Constitutive Relations using Symmetric Positive Definite Neural Networks
• A General Approach to Seismic Inversion with Automatic Differentiation
• Efficient and Robust Parallel DNN Training through Model Parallelism on Multi-GPU Platform
• Efficient PML for the wave equation
• Physics Constrained Learning for Data-driven Inverse Modeling from Sparse Observations
• A Linear Algebraic Approach to Model Parallelism in Deep Learning