Training neural networks faster with minimal tuning using pre-computed lists of hyperparameters for NAdamW

Sourabh Medapati, Priya Kasimbeg, Shankar Krishnan +2

If we want to train a neural network using any of the most popular optimization algorithms, we are immediately faced with a dilemma: how to set the various optimization and regular…