OLion: Approaching the Hadamard Ideal by Intersecting Spectral and ${\ell }_{\infty }$ Implicit Biases

Zixiao Wang, Yifei Shen, Huishuai Zhang

Many optimizers can be interpreted as steepest-descent methods under norm-induced geometries, and thus inherit corresponding implicit biases. We introduce \nameA{} (\fullname{}), w…