Stability in Training PINNs for Stiff PDEs: Why Initial Conditions Matter

Abstract

Training physics-informed neural networks (PINNs) on stiff, time-dependent PDEs remains a fundamental challenge due to optimization instabilities and gradient pathologies. Through a series of rigorous ablation studies and Neural Tangent Kernel (NTK) analysis, we identify that the exact enforcement of initial conditions (ICs) is a decisive factor in stabilizing the training landscape. We present the first systematic ablation of two core strategies: hard initial-condition constrained transformation and self-adaptive loss weighting. Our findings demonstrate that embedding ICs directly into the network architecture provides an implicit time-marching effect, effectively reducing spectral bias and enabling the solution of highly stiff benchmarks, including sharp transitions and high-frequency coupled systems, primarily under periodic boundary conditions, with a Dirichlet extension reported as an additional robustness check. This work provides a scalable framework for developing reliable and physically-consistent neural solvers for complex mechanical systems.