3 papers
cs.DC2025
Rethinking Dynamic Networks and Heterogeneous Computing with Automatic Parallelization
Ruilong Wu, Xinjiao Li, Yisu Wang +2
Hybrid parallelism techniques are essential for efficiently training large language models (LLMs). Nevertheless, current automatic parallel planning frameworks often overlook the s…
cs.DC2025
PacTrain: Pruning and Adaptive Sparse Gradient Compression for Efficient Collective Communication in Distributed Deep Learning
Yisu Wang, Ruilong Wu, Xinjiao Li +1
Large-scale deep neural networks (DNN) exhibit excellent performance for various tasks. As DNNs and datasets grow, distributed training becomes extremely time-consuming and demands…
cs.AR2024
Affordable HPC: Leveraging Small Clusters for Big Data and Graph Computing
Ruilong Wu, Yisu Wang, Dirk Kutscher
This study explores strategies for academic researchers to optimize computational resources within limited budgets, focusing on building small, efficient computing clusters. It del…