3 citations · 4 across the 11 of their papers we have counts for
12 papers
Layer-Parallel Training for Transformers
Shuai Jiang, Marc Salvadó-Benasco, Eric C. Cyr +3
We present a new training methodology for transformers using a multilevel, layer-parallel approach. Through a neural ODE formulation of transformers, our application of a multileve…
Introduction to optimization methods for training SciML models
Alena Kopaničáková, Elisa Riccietti
Optimization is central to both modern machine learning (ML) and scientific machine learning (SciML), yet the structure of the underlying optimization problems differs substantiall…
Multi-Preconditioned LBFGS for Training Finite-Basis PINNs
Marc Salvadó-Benasco, Aymane Kssim, Alexander Heinlein +3
A multi-preconditioned LBFGS (MP-LBFGS) algorithm is introduced for training finite-basis physics-informed neural networks (FBPINNs). The algorithm is motivated by the nonlinear ad…
Trust-Region Methods with Low-Fidelity Objective Models
Andrea Angino, Matteo Aurina, Alena Kopaničáková +3
We introduce two multifidelity trust-region methods based on the Magical Trust Region (MTR) framework. MTR augments the classical trust-region step with a secondary, informative di…
Two-level trust-region method with random subspaces
Andrea Angino, Alena Kopaničáková, Rolf Krause
We introduce a two-level trust-region method (TLTR) for solving unconstrained nonlinear optimization problems. Our method uses a composite iteration step, which is based on two dis…
DeepOnet Based Preconditioning Strategies For Solving Parametric Linear Systems of Equations
Alena Kopaničáková, George Em Karniadakis
We introduce a new class of hybrid preconditioners for solving parametric linear systems of equations. The proposed preconditioners are constructed by hybridizing the deep operator…