Physics-Driven Regularization of Deep Neural Networks for Enhanced Engineering Design and Analysis

Abstract

In this paper, we introduce a physics-driven regularization method for training of deep neural networks (DNNs) for use in engineering design and analysis problems. In particular, we focus on prediction of a physical system, for which in addition to training data, partial or complete information on a set of governing laws is also available. These laws often appear in the form of differential equations, derived from first principles, empirically-validated laws, or domain expertise, and are usually neglected in data-driven prediction of engineering systems. We propose a training approach that utilizes the known governing laws and regularizes data-driven DNN models by penalizing divergence from those laws. The first two numerical examples are synthetic examples, where we show that in constructing a DNN model that best fits the measurements from a physical system, the use of our proposed regularization results in DNNs that are more interpretable with smaller generalization errors, compared to other common regularization methods. The last two examples concern metamodeling for a random Burgers' system and for aerodynamic analysis of passenger vehicles, where we demonstrate that the proposed regularization provides superior generalization accuracy compared to other common alternatives.

References in corpus (26)

• Adam: A Method for Stochastic Optimization
• Explaining and Harnessing Adversarial Examples
• Improving neural networks by preventing co-adaptation of feature detectors
• ADADELTA: An Adaptive Learning Rate Method
• Spectral Normalization for Generative Adversarial Networks
• Artificial Neural Networks for Solving Ordinary and Partial Differential Equations
• DGM: A deep learning algorithm for solving partial differential equations
• Conditional Image Generation with PixelCNN Decoders
• Solving high-dimensional partial differential equations using deep learning
• Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equations
• Deep Convolutional Neural Networks and Data Augmentation for Environmental Sound Classification
• NIPS 2016 Tutorial: Generative Adversarial Networks
• Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
• A unified deep artificial neural network approach to partial differential equations in complex geometries
• Towards Deep Neural Network Architectures Robust to Adversarial Examples
• Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations
• Deep Learning of Vortex Induced Vibrations
• Data Driven Governing Equations Approximation Using Deep Neural Networks
• Dropout Training as Adaptive Regularization
• Deep learning and the Schrödinger equation
• Deep Learning for Accelerated Reliability Analysis of Infrastructure Networks
• A Deep Neural Network Surrogate for High-Dimensional Random Partial Differential Equations
• Numerical Aspects for Approximating Governing Equations Using Data
• Lat-Net: Compressing Lattice Boltzmann Flow Simulations using Deep Neural Networks
• Learning A Physical Long-term Predictor
• Solving Poisson's Equation using Deep Learning in Particle Simulation of PN Junction

Cited by in corpus (14)

• Surrogate Modeling for Fluid Flows Based on Physics-Constrained Deep Learning Without Simulation Data
• Regularized Deep Networks in Intelligent Transportation Systems: A Taxonomy and a Case Study
• Physics-Informed Neural Networks for Multiphysics Data Assimilation with Application to Subsurface Transport
• CAN-PINN: A Fast Physics-Informed Neural Network Based on Coupled-Automatic-Numerical Differentiation Method
• Efficient training of physics-informed neural networks via importance sampling
• Learning in Sinusoidal Spaces with Physics-Informed Neural Networks
• Untrained physically informed neural network for image reconstruction of magnetic field sources
• GRIDS-Net: Inverse shape design and identification of scatterers via geometric regularization and physics-embedded deep learning
• Improved Surrogate Modeling of Fluid Dynamics with Physics-Informed Neural Networks
• Structure Preservation for the Deep Neural Network Multigrid Solver
• On the convergence of PINNs
• Frequency-compensated PINNs for Fluid-dynamic Design Problems
• Description of Structural Biases and Associated Data in Sensor-Rich Environments
• NuGraph2 with Context-Aware Inputs: Physics-Inspired Improvements in Semantic Segmentation