2 papers
cs.LG2025
Generalizing Adam to Manifolds for Efficiently Training Transformers
Benedikt Brantner
One of the primary reasons behind the success of neural networks has been the emergence of an array of new, highly-successful optimizers, perhaps most importantly the Adam optimize…
math.NA2024
Volume-Preserving Transformers for Learning Time Series Data with Structure
Benedikt Brantner, Guillaume de Romemont, Michael Kraus +1
Two of the many trends in neural network research of the past few years have been (i) the learning of dynamical systems, especially with recurrent neural networks such as long shor…