117 citations · 411 across the 25 of their papers we have counts for
8 papers · 1 filter
On the Discrepancy between the Theoretical Analysis and Practical Implementations of Compressed Communication for Distributed Deep Learning
Aritra Dutta, El Houcine Bergou, Ahmed M. Abdelmoniem +4
Compressed communication, in the form of sparsification or quantization of stochastic gradients, is employed to reduce communication costs in distributed data-parallel training of…
Assise: Performance and Availability via NVM Colocation in a Distributed File System
Thomas E. Anderson, Marco Canini, Jongyul Kim +6
The adoption of very low latency persistent memory modules (PMMs) upends the long-established model of disaggregated file system access. Instead, by colocating computation and PMM…
Direct Nonlinear Acceleration
Aritra Dutta, El Houcine Bergou, Yunming Xiao +2
Optimization acceleration techniques such as momentum play a key role in state-of-the-art machine learning algorithms. Recently, generic vector sequence extrapolation techniques, s…
Natural Compression for Distributed Deep Learning
Samuel Horvath, Chen-Yu Ho, Ludovit Horvath +3
Modern deep learning models are often trained in parallel over a collection of distributed machines to reduce training time. In such settings, communication of model updates among…
Consistent SDNs through Network State Fuzzing
Apoorv Shukla, Said Jawad Saidi, Stefan Schmid +3
The conventional wisdom is that a software-defined network (SDN) operates under the premise that the logically centralized control plane has an accurate representation of the actua…
Scaling Distributed Machine Learning with In-Network Aggregation
Amedeo Sapio, Marco Canini, Chen-Yu Ho +7
Training machine learning models in parallel is an increasingly important workload. We accelerate distributed parallel training by designing a communication primitive that uses a p…