53 citations · 70 across the 14 of their papers we have counts for
15 papers
Layer-Parallel Training for Transformers
Shuai Jiang, Marc Salvadó-Benasco, Eric C. Cyr +3
We present a new training methodology for transformers using a multilevel, layer-parallel approach. Through a neural ODE formulation of transformers, our application of a multileve…
Introduction to optimization methods for training SciML models
Alena Kopaničáková, Elisa Riccietti
Optimization is central to both modern machine learning (ML) and scientific machine learning (SciML), yet the structure of the underlying optimization problems differs substantiall…
Multi-Preconditioned LBFGS for Training Finite-Basis PINNs
Marc Salvadó-Benasco, Aymane Kssim, Alexander Heinlein +3
A multi-preconditioned LBFGS (MP-LBFGS) algorithm is introduced for training finite-basis physics-informed neural networks (FBPINNs). The algorithm is motivated by the nonlinear ad…
Trust-Region Methods with Low-Fidelity Objective Models
Andrea Angino, Matteo Aurina, Alena Kopaničáková +3
We introduce two multifidelity trust-region methods based on the Magical Trust Region (MTR) framework. MTR augments the classical trust-region step with a secondary, informative di…
Two-level trust-region method with random subspaces
Andrea Angino, Alena Kopaničáková, Rolf Krause
We introduce a two-level trust-region method (TLTR) for solving unconstrained nonlinear optimization problems. Our method uses a composite iteration step, which is based on two dis…
DeepOnet Based Preconditioning Strategies For Solving Parametric Linear Systems of Equations
Alena Kopaničáková, George Em Karniadakis
We introduce a new class of hybrid preconditioners for solving parametric linear systems of equations. The proposed preconditioners are constructed by hybridizing the deep operator…
Parallel Trust-Region Approaches in Neural Network Training: Beyond Traditional Methods
Ken Trotti, Samuel A. Cruz Alegría, Alena Kopaničáková +1
We propose to train neural networks (NNs) using a novel variant of the ``Additively Preconditioned Trust-region Strategy'' (APTS). The proposed method is based on a parallelizable…
Multilevel Objective-Function-Free Optimization with an Application to Neural Networks Training
S. Gratton, A. Kopanicakova, Ph. L. Toint
A class of multi-level algorithms for unconstrained nonlinear optimization is presented which does not require the evaluation of the objective function. The class contains the mome…
Training of deep residual networks with stochastic MG/OPT
Cyrill von Planta, Alena Kopanicakova, Rolf Krause
We train deep residual networks with a stochastic variant of the nonlinear multigrid method MG/OPT. To build the multilevel hierarchy, we use the dynamical systems viewpoint specif…
Globally Convergent Multilevel Training of Deep Residual Networks
Alena Kopaničáková, Rolf Krause
We propose a globally convergent multilevel training method for deep residual networks (ResNets). The devised method can be seen as a novel variant of the recursive multilevel trus…
Multilevel Active-Set Trust-Region (MASTR) Method for Bound Constrained Minimization
Alena Kopaničáková, Rolf Krause
We introduce a novel variant of the recursive multilevel trust-region (RMTR) method, called MASTR. The method is designed for solving non-convex bound-constrained minimization prob…
Large scale simulation of pressure induced phase-field fracture propagation using Utopia
Patrick Zulian, Alena Kopaničáková, Maria Giuseppina Chiara Nestola +4
Non-linear phase field models are increasingly used for the simulation of fracture propagation models. The numerical simulation of fracture networks of realistic size requires the…
A Multilevel Approach to Training
Vanessa Braglia, Alena Kopaničáková, Rolf Krause
We propose a novel training method based on nonlinear multilevel minimization techniques, commonly used for solving discretized large scale partial differential equations. Our mult…
Multilevel Minimization for Deep Residual Networks
Lisa Gaedke-Merzhäuser, Alena Kopaničáková, Rolf Krause
We present a new multilevel minimization framework for the training of deep residual networks (ResNets), which has the potential to significantly reduce training time and effort. O…
Recursive multilevel trust region method with application to fully monolithic phase-field models of brittle fracture
Alena Kopaničáková, Rolf Krause
The simulation of crack initiation and propagation in an elastic material is difficult, as crack paths with complex topologies have to be resolved. Phase-field approach allows to s…