Eva: A General Vectorized Approximation Framework for Second-order Optimization

Lin Zhang, Shaohuai Shi, Bo Li

Second-order optimization algorithms exhibit excellent convergence properties for training deep learning models, but often incur significant computation and memory overheads. This…