425 citations · 1.1k across the 32 of their papers we have counts for
Showing 2021Show all
2 papers · 1 filter
cs.LG2021
Tensor Normal Training for Deep Learning Models
Yi Ren, Donald Goldfarb
Despite the predominant use of first-order methods for training deep learning models, second-order methods, and in particular, natural gradient methods, remain of interest because…
cs.LG2021★ 1 cited
Kronecker-factored Quasi-Newton Methods for Deep Learning
Yi Ren, Achraf Bahamou, Donald Goldfarb
Second-order methods have the capability of accelerating optimization by using much richer curvature information than first-order methods. However, most are impractical for deep le…