Evolutionary Optimization of Physics-Informed Neural Networks: Evo-PINN Frontiers and Opportunities

Abstract

Deep learning models trained on finite data lack a complete understanding of the physical world. On the other hand, physics-informed neural networks (PINNs) are infused with such knowledge through the incorporation of mathematically expressible laws of nature into their training loss function. By complying with physical laws, PINNs provide advantages over purely data-driven models in limited-data regimes and present as a promising route towards Physical AI. This feature has propelled them to the forefront of scientific machine learning, a domain characterized by scarce and costly data. However, the vision of accurate physics-informed learning comes with significant challenges. This work examines PINNs in terms of model optimization and generalization, shedding light on the need for new algorithmic advances to overcome issues pertaining to the training speed, precision, and generalizability of today's PINN models. Of particular interest are gradient-free evolutionary algorithms (EAs) for optimizing the uniquely complex loss landscapes arising in PINN training. Methods synergizing gradient descent and EAs for discovering bespoke neural architectures and balancing multiple terms in physics-informed learning objectives are positioned as important avenues for future research. Another exciting track is to cast EAs as a meta-learner of generalizable PINN models. To substantiate these proposed avenues, we further highlight results from recent literature to showcase the early success of such approaches in addressing the aforementioned challenges in PINN optimization and generalization.

Accepted for publication in IEEE Computational Intelligence Magazine

References in corpus (40)

• DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators
• Machine learning and the physical sciences
• Theory-guided Data Science: A New Paradigm for Scientific Discovery from Data
• B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data
• Adaptive activation functions accelerate convergence in deep and physics-informed neural networks
• Explainable Machine Learning for Scientific Insights and Discoveries
• Informed Machine Learning -- A Taxonomy and Survey of Integrating Knowledge into Learning Systems
• On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks
• Physics-Informed Multi-LSTM Networks for Metamodeling of Nonlinear Structures
• Self-adaptive loss balanced Physics-informed neural networks for the incompressible Navier-Stokes equations
• SciANN: A Keras/Tensorflow wrapper for scientific computations and physics-informed deep learning using artificial neural networks
• CAN-PINN: A Fast Physics-Informed Neural Network Based on Coupled-Automatic-Numerical Differentiation Method
• Physics-Informed Neural Network for Modelling the Thermochemical Curing Process of Composite-Tool Systems During Manufacture
• Ensemble-SINDy: Robust sparse model discovery in the low-data, high-noise limit, with active learning and control
• A Physics Informed Neural Network for Time-Dependent Nonlinear and Higher Order Partial Differential Equations
• Transfer learning based physics-informed neural networks for solving inverse problems in engineering structures under different loading scenarios
• Transfer learning based multi-fidelity physics informed deep neural network
• Local Extreme Learning Machines and Domain Decomposition for Solving Linear and Nonlinear Partial Differential Equations
• Physics Informed Neural Networks for Control Oriented Thermal Modeling of Buildings
• Multi-Objective Loss Balancing for Physics-Informed Deep Learning
• Meta-learning PINN loss functions
• Weak baselines and reporting biases lead to overoptimism in machine learning for fluid-related partial differential equations
• Temperature Field Inversion of Heat-Source Systems via Physics-Informed Neural Networks
• Learning in Sinusoidal Spaces with Physics-Informed Neural Networks
• A novel meta-learning initialization method for physics-informed neural networks
• On Computing the Hyperparameter of Extreme Learning Machines: Algorithm and Application to Computational PDEs, and Comparison with Classical and High-Order Finite Elements
• Data vs. Physics: The Apparent Pareto Front of Physics-Informed Neural Networks
• PINN Training using Biobjective Optimization: The Trade-off between Data Loss and Residual Loss
• Robust quantum gates using smooth pulses and physics-informed neural networks
• Can Transfer Neuroevolution Tractably Solve Your Differential Equations?
• RandONet: Shallow-Networks with Random Projections for learning linear and nonlinear operators
• EvoJAX: Hardware-Accelerated Neuroevolution
• CMA-ES with Learning Rate Adaptation
• Reinforcement learning based adaptive metaheuristics
• A Novel Multi-fidelity Surrogate for Efficient Turbine Design Optimization
• Jack and Masters of all Trades: One-Pass Learning Sets of Model Sets From Large Pre-Trained Models
• SVD-PINNs: Transfer Learning of Physics-Informed Neural Networks via Singular Value Decomposition
• LSA-PINN: Linear Boundary Connectivity Loss for Solving PDEs on Complex Geometry
• Discovering Physics-Informed Neural Networks Model for Solving Partial Differential Equations through Evolutionary Computation
• Neuroevolution of Physics-Informed Neural Nets: Benchmark Problems and Comparative Results