On the Principles Behind Neural Network Optimizers

Yushun Zhang

Reliable optimization is central to neural network (NN) training, yet Adam, the default optimizer for modern LLMs, rests on a fragile foundation. This thesis develops a principled…