354 citations · 693 across the 36 of their papers we have counts for
5 papers · 1 filter
GradiVeQ: Vector Quantization for Bandwidth-Efficient Gradient Aggregation in Distributed CNN Training
Mingchao Yu, Zhifeng Lin, Krishna Narra +6
Data parallelism can boost the training speed of convolutional neural networks (CNN), but could suffer from significant communication costs caused by gradient aggregation. To allev…
Pipe-SGD: A Decentralized Pipelined SGD Framework for Distributed Deep Net Training
Youjie Li, Mingchao Yu, Songze Li +3
Distributed training of deep nets is an important technique to address some of the present day computing challenges like memory consumption and computational demands. Classical dis…
PolyShard: Coded Sharding Achieves Linearly Scaling Efficiency and Security Simultaneously
Songze Li, Mingchao Yu, Chien-Sheng Yang +3
Today's blockchain designs suffer from a trilemma claiming that no blockchain system can simultaneously achieve decentralization, security, and performance scalability. For current…
Polynomially Coded Regression: Optimal Straggler Mitigation via Data Encoding
Songze Li, Seyed Mohammadreza Mousavi Kalan, Qian Yu +2
We consider the problem of training a least-squares regression model on a large dataset using gradient descent. The computation is carried out on a distributed system consisting of…
Compressed Coded Distributed Computing
Songze Li, Mohammad Ali Maddah-Ali, A. Salman Avestimehr
Communication overhead is one of the major performance bottlenecks in large-scale distributed computing systems, in particular for machine learning applications. Conventionally, co…