Do physics-informed neural networks (PINNs) need to be deep? Shallow PINNs using the Levenberg-Marquardt algorithm

Abstract

This work investigates shallow physics-informed neural networks (PINNs) for solving forward and inverse problems governed by nonlinear partial differential equations (PDEs). By formulating PINN training as a nonlinear least-squares problem, the Levenberg-Marquardt (LM) algorithm is used to efficiently optimize the network parameters. Exact analytical expressions for neural-network derivatives with respect to the input variables are derived, revealing the relationships between the network output and its spatial and temporal derivatives and providing a clearer interpretation of the PINN architecture. These expressions are then used to derive explicit formulas for the Jacobian matrix required by LM. The proposed approach is evaluated on the Burgers, Schrödinger, Allen-Cahn, and three-dimensional Bratu equations. Numerical results show that LM substantially outperforms BFGS, L-BFGS, and Adam in convergence speed, accuracy, and final loss values. Comparisons with deeper networks further demonstrate that shallow LM-PINNs can achieve higher accuracy with substantially fewer parameters, emphasizing the importance of considering network architecture and optimization strategy jointly. The explicit analytical Jacobian also provides computational and memory advantages that are particularly relevant to large-scale PINNs. Overall, these results suggest that, for a broad class of PDEs, shallow PINNs combined with effective second-order optimization can provide accurate and computationally efficient solutions to both forward and inverse problems.