4 papers
Curvature-Aware Optimization for High-Accuracy Physics-Informed Neural Networks
Anas Jnini, Elham Kiyani, Khemraj Shukla +5
Efficient and robust optimization is essential for neural networks, enabling scientific machine learning models to converge rapidly to very high accuracy -- faithfully capturing co…
Gauss-Newton Natural Gradient Descent for Shape Learning
James King, Arturs Berzins, Siddhartha Mishra +1
We explore the use of the Gauss-Newton method for optimization in shape learning, including implicit neural surfaces and geometry-informed neural networks. The method addresses key…
Collapsing Taylor Mode Automatic Differentiation
Felix Dangel, Tim Siebert, Marius Zeinhofer +1
Computing partial differential equation (PDE) operators via nested backpropagation is expensive, yet popular, and severely restricts their utility for scientific machine learning.…
Improving Energy Natural Gradient Descent through Woodbury, Momentum, and Randomization
Andrés Guzmán-Cordero, Felix Dangel, Gil Goldshlager +1
Natural gradient methods significantly accelerate the training of Physics-Informed Neural Networks (PINNs), but are often prohibitively costly. We introduce a suite of techniques t…