13 citations · 17 across the 2 of their papers we have counts for
2 papers
cs.DC2023★ 13 cited
PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel
Yanli Zhao, Andrew Gu, Rohan Varma +15
It is widely acknowledged that large models have the potential to deliver superior performance across a broad range of domains. Despite the remarkable progress made in the field of…
cs.LG2021★ 4 cited
Ranger21: a synergistic deep learning optimizer
Less Wright, Nestor Demeure
As optimizers are critical to the performances of neural networks, every year a large number of papers innovating on the subject are published. However, while most of these publica…