4 papers
ViFusion: In-Network Tensor Fusion for Scalable Video Feature Indexing
Yisu Wang, Yixiang Zhu, Xinjiao Li +3
Large-scale video feature indexing in datacenters is critically dependent on efficient data transfer. Although in-network computation has emerged as a compelling strategy for accel…
NetSenseML: Network-Adaptive Compression for Efficient Distributed Machine Learning
Yisu Wang, Xinjiao Li, Ruilong Wu +2
Training large-scale distributed machine learning models imposes considerable demands on network infrastructure, often resulting in sudden traffic spikes that lead to congestion, i…
Rethinking Dynamic Networks and Heterogeneous Computing with Automatic Parallelization
Ruilong Wu, Xinjiao Li, Yisu Wang +2
Hybrid parallelism techniques are essential for efficiently training large language models (LLMs). Nevertheless, current automatic parallel planning frameworks often overlook the s…
PacTrain: Pruning and Adaptive Sparse Gradient Compression for Efficient Collective Communication in Distributed Deep Learning
Yisu Wang, Ruilong Wu, Xinjiao Li +1
Large-scale deep neural networks (DNN) exhibit excellent performance for various tasks. As DNNs and datasets grow, distributed training becomes extremely time-consuming and demands…