A Physical Response-and-Memory Model for Muon Optimization

Yinze Hu, Hongjun Xiang, Xingao Gong +1

Training large language models is costly. How low a loss the same compute can ultimately reach depends on how each step's gradient is converted into a weight update; the rule that…