Transforming physics-informed machine learning to convex optimization

Abstract

Physics-Informed Machine Learning (PIML) offers a powerful paradigm of integrating data with physical laws to address important scientific problems, such as parameter estimation, inferring hidden physics, equation discovery, and state prediction, etc. However, PIML still faces many serious optimization challenges that significantly restrict its applications. In this study, we propose a comprehensive framework that transforms PIML to convex optimization to overcome all these limitations, referred to as Convex-PIML. The linear combination of B-splines is utilized to approximate the data, promoting the convexity of the loss function. By replacing the non-convex components of the loss function with convex approximations, the problem is further converted into a sequence of successively refined approximated convex optimization problems. This conversion allows the use of well-established convex optimization algorithms, obtaining solutions effectively and efficiently. Furthermore, an adaptive knot optimization method based on error estimate is introduced to mitigate the spectral bias issue of PIML, further improving the performance. The proposed theoretically guaranteed framework is tested in scenarios with distinct types of physical prior. The results indicate that optimization problems are effectively solved in these scenarios, highlighting the potential of the framework for broad applications.

33 pages,14 figures

References in corpus (17)

• Adam: A Method for Stochastic Optimization
• Discovering governing equations from data: Sparse identification of nonlinear dynamical systems
• DGM: A deep learning algorithm for solving partial differential equations
• Neural Tangent Kernel: Convergence and Generalization in Neural Networks
• On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks
• A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks
• Finite Basis Physics-Informed Neural Networks (FBPINNs): a scalable domain decomposition approach for solving differential equations
• Non-convex Optimization for Machine Learning
• Decomposition by Partial Linearization: Parallel Optimization of Multi-Agent Systems
• Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent
• Transfer learning based physics-informed neural networks for solving inverse problems in engineering structures under different loading scenarios
• The Convergence Rate of Neural Networks for Learned Functions of Different Frequencies
• A Unified Successive Pseudo-Convex Approximation Framework
• Inexact Block Coordinate Descent Algorithms for Nonsmooth Nonconvex Optimization
• SoftAdapt: Techniques for Adaptive Loss Weighting of Neural Networks with Multi-Part Loss Functions
• PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers
• Convex Physics Informed Neural Networks for the Monge-Ampère Optimal Transport Problem