15 citations · 21 across the 6 of their papers we have counts for
6 papers · 1 filter
VirtualFlow: Decoupling Deep Learning Models from the Underlying Hardware
Andrew Or, Haoyu Zhang, Michael J. Freedman
State-of-the-art deep learning systems such as TensorFlow and PyTorch tightly couple the model with the underlying hardware. This coupling requires the user to modify application l…
Scaling Distributed Deep Learning Workloads beyond the Memory Capacity with KARMA
Mohamed Wahib, Haoyu Zhang, Truong Thao Nguyen +5
The dedicated memory of hardware accelerators can be insufficient to store all weights and/or intermediate states of large deep learning models. Although model parallelism is a via…
A Study of Single and Multi-device Synchronization Methods in Nvidia GPUs
Lingqi Zhang, Mohamed Wahib, Haoyu Zhang +1
GPUs are playing an increasingly important role in general-purpose computing. Many algorithms require synchronizations at different levels of granularity in a single GPU. Additiona…
Double-precision FPUs in High-Performance Computing: an Embarrassment of Riches?
Jens Domke, Kazuaki Matsumura, Mohamed Wahib +6
Among the (uncontended) common wisdom in High-Performance Computing (HPC) is the applications' need for large amount of double-precision support in hardware. Hardware manufacturers…
NetChain: Scale-Free Sub-RTT Coordination (Extended Version)
Xin Jin, Xiaozhou Li, Haoyu Zhang +5
Coordination services are a fundamental building block of modern cloud systems, providing critical functionalities like configuration management and distributed locking. The major…
SLAQ: Quality-Driven Scheduling for Distributed Machine Learning
Haoyu Zhang, Logan Stafman, Andrew Or +1
Training machine learning (ML) models with large datasets can incur significant resource contention on shared clusters. This training typically involves many iterations that contin…