32 citations · 74 across the 4 of their papers we have counts for
Showing cs.DCShow all
2 papers · 1 filter
cs.DC2021★ 20 cited
Maximizing Parallelism in Distributed Training for Huge Neural Networks
Zhengda Bian, Qifan Xu, Boxiang Wang +1
The recent Natural Language Processing techniques have been refreshing the state-of-the-art performance at an incredible speed. Training huge language models is therefore an impera…
cs.DC2021★ 32 cited
Tesseract: Parallelize the Tensor Parallelism Efficiently
Boxiang Wang, Qifan Xu, Zhengda Bian +1
Together with the improvements in state-of-the-art accuracies of various tasks, deep learning models are getting significantly larger. However, it is extremely difficult to impleme…