Data-Parallel Neural Network Training via Nonlinearly Preconditioned Trust-Region Method

Samuel A. Cruz Alegría, Ken Trotti, Alena Kopaničáková +1

Parallel training methods are increasingly relevant in machine learning (ML) due to the continuing growth in model and dataset sizes. We propose a variant of the Additively Precond…