1 citations · 1 across the 2 of their papers we have counts for
2 papers
cs.LG2023
Jorge: Approximate Preconditioning for GPU-efficient Second-order Optimization
Siddharth Singh, Zachary Sating, Abhinav Bhatele
Despite their better convergence properties compared to first-order optimizers, second-order optimizers for deep learning have been less popular due to their significant computatio…
cs.LG2023★ 1 cited
A 4D Hybrid Algorithm to Scale Parallel Training to Thousands of GPUs
Siddharth Singh, Prajwal Singhania, Aditya K. Ranjan +2
Heavy communication, in particular, collective operations, can become a critical performance bottleneck in scaling the training of billion-parameter neural networks to large-scale…