Highly-scalable, physics-informed GANs for learning solutions of stochastic PDEs

Abstract

Uncertainty quantification for forward and inverse problems is a central challenge across physical and biomedical disciplines. We address this challenge for the problem of modeling subsurface flow at the Hanford Site by combining stochastic computational models with observational data using physics-informed GAN models. The geographic extent, spatial heterogeneity, and multiple correlation length scales of the Hanford Site require training a computationally intensive GAN model to thousands of dimensions. We develop a hierarchical scheme for exploiting domain parallelism, map discriminators and generators to multiple GPUs, and employ efficient communication schemes to ensure training stability and convergence. We developed a highly optimized implementation of this scheme that scales to 27,500 NVIDIA Volta GPUs and 4584 nodes on the Summit supercomputer with a 93.1% scaling efficiency, achieving peak and sustained half-precision rates of 1228 PF/s and 1207 PF/s.

arXiv comment3rd Deep Learning on Supercomputers Workshop (DLS) at SC19

References in corpus (11)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Deep Learning using Rectified Linear Units (ReLU)
• Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equations
• Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations
• Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
• Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems
• Horovod: fast and easy distributed deep learning in TensorFlow
• Adversarial Uncertainty Quantification in Physics-Informed Neural Networks
• Highly Scalable Deep Learning Training System with Mixed-Precision: Training ImageNet in Four Minutes
• Learning Parameters and Constitutive Relationships with Physics Informed Deep Neural Networks

Cited by in corpus (6)

• Reconstruction of turbulent data with deep generative models for semantic inpainting from TURB-Rot database
• Physics-Informed Machine Learning Method for Large-Scale Data Assimilation Problems
• PCNN: A physics-constrained neural network for multiphase flows
• Smart-PGSim: Using Neural Network to Accelerate AC-OPF Power Grid Simulation
• Inverse Estimation of Elastic Modulus Using Physics-Informed Generative Adversarial Networks
• A Linear Algebraic Approach to Model Parallelism in Deep Learning