4 papers
Bundled References: An Abstraction for Highly-Concurrent Linearizable Range Queries
Jacob Nelson, Ahmed Hassan, Roberto Palmieri
We present bundled references, a new building block to provide linearizable range query operations for highly concurrent linked data structures. Bundled references allow range quer…
Synthesizing Optimal Collective Algorithms
Zixian Cai, Zhengyang Liu, Saeed Maleki +4
Collective communication algorithms are an important component of distributed computation. Indeed, in the case of deep-learning, collective communication is the Amdahl's bottleneck…
Scaling Distributed Machine Learning with In-Network Aggregation
Amedeo Sapio, Marco Canini, Chen-Yu Ho +7
Training machine learning models in parallel is an increasingly important workload. We accelerate distributed parallel training by designing a communication primitive that uses a p…
Parameter Hub: a Rack-Scale Parameter Server for Distributed Deep Neural Network Training
Liang Luo, Jacob Nelson, Luis Ceze +2
Distributed deep neural network (DDNN) training constitutes an increasingly important workload that frequently runs in the cloud. Larger DNN models and faster compute engines are s…