A Dual-Dimer Method for Training Physics-Constrained Neural Networks with Minimax Architecture

Abstract

Data sparsity is a common issue to train machine learning tools such as neural networks for engineering and scientific applications, where experiments and simulations are expensive. Recently physics-constrained neural networks (PCNNs) were developed to reduce the required amount of training data. However, the weights of different losses from data and physical constraints are adjusted empirically in PCNNs. In this paper, a new physics-constrained neural network with the minimax architecture (PCNN-MM) is proposed so that the weights of different losses can be adjusted systematically. The training of the PCNN-MM is searching the high-order saddle points of the objective function. A novel saddle point search algorithm called Dual-Dimer method is developed. It is demonstrated that the Dual-Dimer method is computationally more efficient than the gradient descent ascent method for nonconvex-nonconcave functions and provides additional eigenvalue information to verify search results. A heat transfer example also shows that the convergence of PCNN-MMs is faster than that of traditional PCNNs.

34 pages, 5 figures, accepted by neural networks

References in corpus (5)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data
• On Finding Local Nash Equilibria (and Only Local Nash Equilibria) in Zero-Sum Games
• Efficient Algorithms for Smooth Minimax Optimization

Cited by in corpus (17)

• Self-adaptive loss balanced Physics-informed neural networks for the incompressible Navier-Stokes equations
• Uncertainty Quantification in Machine Learning for Engineering Design and Health Prognostics: A Tutorial
• Residual-based attention in physics-informed neural networks
• Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanism
• A practical PINN framework for multi-scale problems with multi-magnitude loss terms
• Physics-Informed Neural Networks with Adaptive Localized Artificial Viscosity
• Unveiling the optimization process of Physics Informed Neural Networks: How accurate and competitive can PINNs be?
• Investigating and Mitigating Failure Modes in Physics-informed Neural Networks (PINNs)
• Improved Training of Physics-Informed Neural Networks with Model Ensembles
• Self-adaptive weights based on balanced residual decay rate for physics-informed neural networks and deep operator networks
• Towards Complex Dynamic Physics System Simulation with Graph Neural ODEs
• Deep learning for full-field ultrasonic characterization
• Learning in PINNs: Phase transition, diffusion equilibrium, and generalization
• Splitting physics-informed neural networks for inferring the dynamics of integer- and fractional-order neuron models
• Sobolev neural network with residual weighting as a surrogate in linear and non-linear mechanics
• Fine-Tuning Hybrid Physics-Informed Neural Networks for Vehicle Dynamics Model Estimation
• Navigating Uncertainties in Machine Learning for Structural Dynamics: A Comprehensive Survey of Probabilistic and Non-Probabilistic Approaches in Forward and Inverse Problems