36 citations · 63 across the 11 of their papers we have counts for
7 papers · 1 filter
TrainVerify: Equivalence-Based Verification for Distributed LLM Training
Yunchi Lu, Youshan Miao, Cheng Tan +4
Training large language models (LLMs) at scale requires parallel execution across thousands of devices, incurring enormous computational costs. Yet, these costly distributed traini…
Tessel: Boosting Distributed Execution of Large DNN Models via Flexible Schedule Search
Zhiqi Lin, Youshan Miao, Guanbin Xu +4
Increasingly complex and diverse deep neural network (DNN) models necessitate distributing the execution across multiple devices for training and inference tasks, and also require…
SuperScaler: Supporting Flexible DNN Parallelization via a Unified Abstraction
Zhiqi Lin, Youshan Miao, Guodong Liu +10
With the growing model size, deep neural networks (DNN) are increasingly trained over massive GPU accelerators, which demands a proper parallelization plan that transforms a DNN mo…
Breaking the Computation and Communication Abstraction Barrier in Distributed Machine Learning Workloads
Abhinav Jangda, Jun Huang, Guodong Liu +6
Recent trend towards increasing large machine learning models require both training and inference tasks to be distributed. Considering the huge cost of training these models, it is…
CrossoverScheduler: Overlapping Multiple Distributed Training Applications in a Crossover Manner
Cheng Luo, Lei Qu, Youshan Miao +2
Distributed deep learning workloads include throughput-intensive training tasks on the GPU clusters, where the Distributed Stochastic Gradient Descent (SGD) incurs significant comm…
Towards Efficient Large-Scale Graph Neural Network Computing
Lingxiao Ma, Zhi Yang, Youshan Miao +4
Recent deep learning models have moved beyond low-dimensional regular grids such as image, video, and speech, to high-dimensional graph-structured data, such as social networks, br…
RPC Considered Harmful: Fast Distributed Deep Learning on RDMA
Jilong Xue, Youshan Miao, Cheng Chen +3
Deep learning emerges as an important new resource-intensive workload and has been successfully applied in computer vision, speech, natural language processing, and so on. Distribu…