47 citations · 138 across the 18 of their papers we have counts for
Showing cs.DCShow all
2 papers · 1 filter
cs.DC2023★ 6 cited
SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient
Max Ryabinin, Tim Dettmers, Michael Diskin +1
Many deep learning applications benefit from using large models with billions of parameters. Training these models is notoriously expensive due to the need for specialized HPC clus…
cs.DC2020★ 17 cited
Towards Crowdsourced Training of Large Neural Networks using Decentralized Mixture-of-Experts
Max Ryabinin, Anton Gusev
Many recent breakthroughs in deep learning were achieved by training increasingly larger models on massive datasets. However, training such models can be prohibitively expensive. F…