Schattor: Schatten-family methods for deep learning optimization

Bohao Ma, Junyu Zhang, Chuan He

Modern deep learning optimization features heterogeneous parameter structures, noisy gradients, and highly nonconvex landscapes, posing significant challenges for both algorithm de…