16.2k citations · 16.4k across the 15 of their papers we have counts for
Showing cs.DCShow all
2 papers · 1 filter
cs.DC2023★ 13 cited
PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel
Yanli Zhao, Andrew Gu, Rohan Varma +15
It is widely acknowledged that large models have the potential to deliver superior performance across a broad range of domains. Despite the remarkable progress made in the field of…
cs.DC2021★ 3 cited
Power Consumption Analysis of Parallel Algorithms on GPUs
Frédéric Magoulès, Abal-Kassim Cheik Ahamed, Alban Desmaison +4
Due to their highly parallel multi-cores architecture, GPUs are being increasingly used in a wide range of computationally intensive applications. Compared to CPUs, GPUs can achiev…