855 citations · 3.9k across the 142 of their papers we have counts for
10 papers · 1 filter
GradiVeQ: Vector Quantization for Bandwidth-Efficient Gradient Aggregation in Distributed CNN Training
Mingchao Yu, Zhifeng Lin, Krishna Narra +6
Data parallelism can boost the training speed of convolutional neural networks (CNN), but could suffer from significant communication costs caused by gradient aggregation. To allev…
Pipe-SGD: A Decentralized Pipelined SGD Framework for Distributed Deep Net Training
Youjie Li, Mingchao Yu, Songze Li +3
Distributed training of deep nets is an important technique to address some of the present day computing challenges like memory consumption and computational demands. Classical dis…
PolyShard: Coded Sharding Achieves Linearly Scaling Efficiency and Security Simultaneously
Songze Li, Mingchao Yu, Chien-Sheng Yang +3
Today's blockchain designs suffer from a trilemma claiming that no blockchain system can simultaneously achieve decentralization, security, and performance scalability. For current…
Lagrange Coded Computing: Optimal Design for Resiliency, Security and Privacy
Qian Yu, Songze Li, Netanel Raviv +3
We consider a scenario involving computations over a massive dataset stored distributedly across multiple workers, which is at the core of distributed learning algorithms. We propo…
Polynomially Coded Regression: Optimal Straggler Mitigation via Data Encoding
Songze Li, Seyed Mohammadreza Mousavi Kalan, Qian Yu +2
We consider the problem of training a least-squares regression model on a large dataset using gradient descent. The computation is carried out on a distributed system consisting of…
Compressed Coded Distributed Computing
Songze Li, Mohammad Ali Maddah-Ali, A. Salman Avestimehr
Communication overhead is one of the major performance bottlenecks in large-scale distributed computing systems, in particular for machine learning applications. Conventionally, co…