Efficient PINNs via Multi-Head Unimodular Regularization of the Solutions Space

Abstract

Non-linear differential equations are a fundamental tool to describe different phenomena in nature. However, we still lack a well-established method to tackle stiff differential equations. Here we present a machine learning framework to facilitate the solution of nonlinear multiscale differential equations and, especially, inverse problems using Physics-Informed Neural Networks (PINNs). This framework is based on what is called \textit{multi-head} (MH) training, which involves training the network to learn a general space of all solutions for a given set of equations with certain variability, rather than learning a specific solution of the system. This setup is used with a second novel technique that we call Unimodular Regularization (UR) of the latent space of solutions. We show that the multi-head approach, combined with Unimodular Regularization, significantly improves the efficiency of PINNs by facilitating the transfer learning process thereby enabling the finding of solutions for nonlinear, coupled, and multiscale differential equations.

References in corpus (21)

• DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators
• DGM: A deep learning algorithm for solving partial differential equations
• Fourier Neural Operator for Parametric Partial Differential Equations
• Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data
• Hamiltonian Neural Networks
• Mimicking the QCD equation of state with a dual black hole
• Hamiltonian neural networks for solving equations of motion
• Unimodular Gravity vs General Relativity: A status report
• Physics enhanced neural networks predict order and chaos
• $\Delta$-PINNs: physics-informed neural networks on complex geometries
• Heating up Exotic RG Flows with Holography
• Solving Differential Equations Using Neural Network Solution Bundles
• Gravitational Duals from Equations of State
• Transfer Learning with Physics-Informed Neural Networks for Efficient Simulation of Branched Flows
• Invariant Physics-Informed Neural Networks for Ordinary Differential Equations
• Thoughts on the Consistency between Ricci Flow and Neural Network Behavior
• Data-Guided Physics-Informed Neural Networks for Solving Inverse Problems in Partial Differential Equations
• A Primer on Unimodular Gravity
• RicciNets: Curvature-guided Pruning of High-performance Neural Networks Using Ricci Flow
• One-Shot Transfer Learning for Nonlinear ODEs
• Stiff Transfer Learning for Physics-Informed Neural Networks