Showing cs.LGShow all
2 papers · 1 filter
cs.LG2023
Eva: A General Vectorized Approximation Framework for Second-order Optimization
Lin Zhang, Shaohuai Shi, Bo Li
Second-order optimization algorithms exhibit excellent convergence properties for training deep learning models, but often incur significant computation and memory overheads. This…
cs.LG2022
Scalable K-FAC Training for Deep Neural Networks with Distributed Preconditioning
Lin Zhang, Shaohuai Shi, Wei Wang +1
The second-order optimization methods, notably the D-KFAC (Distributed Kronecker Factored Approximate Curvature) algorithms, have gained traction on accelerating deep neural networ…