Training multi-objective/multi-task collocation physics-informed neural network with student/teachers transfer learnings

Abstract

This paper presents a PINN training framework that employs (1) pre-training steps that accelerates and improve the robustness of the training of physics-informed neural network with auxiliary data stored in point clouds, (2) a net-to-net knowledge transfer algorithm that improves the weight initialization of the neural network and (3) a multi-objective optimization algorithm that may improve the performance of a physical-informed neural network with competing constraints. We consider the training and transfer and multi-task learning of physics-informed neural network (PINN) as multi-objective problems where the physics constraints such as the governing equation, boundary conditions, thermodynamic inequality, symmetry, and invariant properties, as well as point cloud used for pre-training can sometimes lead to conflicts and necessitating the seek of the Pareto optimal solution. In these situations, weighted norms commonly used to handle multiple constraints may lead to poor performance, while other multi-objective algorithms may scale poorly with increasing dimensionality. To overcome this technical barrier, we adopt the concept of vectorized objective function and modify a gradient descent approach to handle the issue of conflicting gradients. Numerical experiments are compared the benchmark boundary value problems solved via PINN. The performance of the proposed paradigm is compared against the classical equal-weighted norm approach. Our numerical experiments indicate that the brittleness and lack of robustness demonstrated in some PINN implementations can be overcome with the proposed strategy.

References in corpus (27)

• Adam: A Method for Stochastic Optimization
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• An Overview of Multi-Task Learning in Deep Neural Networks
• DGM: A deep learning algorithm for solving partial differential equations
• An Energy Approach to the Solution of Partial Differential Equations in Computational Mechanics via Machine Learning: Concepts, Implementation and Applications
• Turbulence Modeling in the Age of Data
• Adaptive activation functions accelerate convergence in deep and physics-informed neural networks
• Fourier Neural Operator for Parametric Partial Differential Equations
• Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data
• hp-VPINNs: Variational Physics-Informed Neural Networks With Domain Decomposition
• On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks
• PPINN: Parareal Physics-Informed Neural Network for time-dependent PDEs
• Lift & Learn: Physics-informed machine learning for large-scale nonlinear dynamical systems
• Artificial Neural Network Methods in Quantum Mechanics
• Net2Net: Accelerating Learning via Knowledge Transfer
• On the Spectral Bias of Neural Networks
• The Neural Particle Method -- An Updated Lagrangian Physics Informed Neural Network for Computational Fluid Dynamics
• Meta-learning PINN loss functions
• A Generalized Algorithm for Multi-Objective Reinforcement Learning and Policy Adaptation
• Gradient Surgery for Multi-Task Learning
• Learning Parameters and Constitutive Relationships with Physics Informed Deep Neural Networks
• Methodology study of machine learning for the neutron star equation of state
• The Neural Network Approach to Inverse Problems in Differential Equations
• Adaptive Scheduling for Multi-Task Learning
• NVIDIA SimNet^{TM}: an AI-accelerated multi-physics simulation framework
• Exact imposition of boundary conditions with distance functions in physics-informed deep neural networks
• MIM: A deep mixed residual method for solving high-order partial differential equations

Cited by in corpus (5)

• Transfer learning based physics-informed neural networks for solving inverse problems in engineering structures under different loading scenarios
• Learning in Sinusoidal Spaces with Physics-Informed Neural Networks
• Geometric deep learning for computational mechanics Part II: Graph embedding for interpretable multiscale plasticity
• Manifold embedding data-driven mechanics
• A Data-driven Multi-fidelity Physics-informed Learning Framework for Smart Manufacturing: A Composites Processing Case Study