5 papers
Introduction to optimization methods for training SciML models
Alena KopaniÄáková, Elisa Riccietti
Optimization is central to both modern machine learning (ML) and scientific machine learning (SciML), yet the structure of the underlying optimization problems differs substantiall…
Multi-Preconditioned LBFGS for Training Finite-Basis PINNs
Marc Salvadó-Benasco, Aymane Kssim, Alexander Heinlein +3
A multi-preconditioned LBFGS (MP-LBFGS) algorithm is introduced for training finite-basis physics-informed neural networks (FBPINNs). The algorithm is motivated by the nonlinear ad…
Layer-Parallel Training for Transformers
Shuai Jiang, Marc Salvadó-Benasco, Eric C. Cyr +3
We present a new training methodology for transformers using a multilevel, layer-parallel approach. Through a neural ODE formulation of transformers, our application of a multileve…
Trust-Region Methods with Low-Fidelity Objective Models
Andrea Angino, Matteo Aurina, Alena KopaniÄáková +3
We introduce two multifidelity trust-region methods based on the Magical Trust Region (MTR) framework. MTR augments the classical trust-region step with a secondary, informative di…
Two-level trust-region method with random subspaces
Andrea Angino, Alena KopaniÄáková, Rolf Krause
We introduce a two-level trust-region method (TLTR) for solving unconstrained nonlinear optimization problems. Our method uses a composite iteration step, which is based on two dis…