2 citations · 5 across the 8 of their papers we have counts for
Showing cs.DCShow all
3 papers · 1 filter
cs.DC2023★ 1 cited
Oases: Efficient Large-Scale Model Training on Commodity Servers via Overlapped and Automated Tensor Model Parallelism
Shengwei Li, Zhiquan Lai, Dongsheng Li +5
Deep learning is experiencing a rise in large-scale models. Training large-scale models is costly, prompting researchers to train large-scale models on commodity servers that more…
cs.DC2021★ 1 cited
Communication-Efficient Distributed Learning via Sparse and Adaptive Stochastic Gradient
Xiaoge Deng, Dongsheng Li, Tao Sun +1
Gradient-based optimization methods implemented on distributed computing architectures are increasingly used to tackle large-scale machine learning applications. A key bottleneck i…
cs.DC2021★ 1 cited
S2 Reducer: High-Performance Sparse Communication to Accelerate Distributed Deep Learning
Keshi Ge, Yongquan Fu, Zhiquan Lai +2
Distributed stochastic gradient descent (SGD) approach has been widely used in large-scale deep learning, and the gradient collective method is vital to ensure the training scalabi…