51 citations · 105 across the 3 of their papers we have counts for
3 papers
cs.LG2023★ 20 cited
AlpaServe: Statistical Multiplexing with Model Parallelism for Deep Learning Serving
Zhuohan Li, Lianmin Zheng, Yinmin Zhong +8
Model parallelism is conventionally viewed as a method to scale a single large deep learning model beyond the memory limits of a single device. In this paper, we demonstrate that m…
cs.LG2018★ 34 cited
Large batch size training of neural networks with adversarial training and second-order information
Zhewei Yao, Amir Gholami, Daiyaan Arfeen +4
The most straightforward method to accelerate Stochastic Gradient Descent (SGD) computation is to distribute the randomly selected batch of inputs over multiple processors. To keep…
cs.CV2018★ 51 cited
Deep Mixture of Experts via Shallow Embedding
Xin Wang, Fisher Yu, Lisa Dunlap +5
Larger networks generally have greater representational power at the cost of increased computational complexity. Sparsifying such networks has been an active area of research but h…