9 citations · 17 across the 8 of their papers we have counts for
5 papers · 1 filter
MegaScale-Omni: A Hyper-Scale, Workload-Resilient System for MultiModal LLM Training in Production
Chunyu Xue, Yangrui Chen, Jianyu Jiang +14
As the foundational component of versatile AI applications, training an multimodal large language model (MLLM) relies on multimodal datasets with dynamic modality mixture proportio…
Mesh-Attention: A New Communication-Efficient Distributed Attention with Improved Data Locality
Sirui Chen, Jingji Chen, Siqi Zhu +3
Distributed attention is essential for scaling large language models (LLMs) to long contexts, yet existing methods either have limited parallelism or incur high communication costs…
GNNPipe: Scaling Deep GNN Training with Pipelined Model Parallelism
Jingji Chen, Zhuoming Chen, Xuehai Qian
Communication is a key bottleneck for distributed graph neural network (GNN) training. This paper proposes GNNPipe, a new approach that scales the distributed full-graph deep GNN t…
Kudu: An Efficient and Scalable Distributed Graph Pattern Mining Engine
Jingji Chen, Xuehai Qian
This paper proposes Kudu, a distributed execution engine with a well-defined abstraction that can be integrated with existing single-machine graph pattern mining (GPM) systems to p…
DwarvesGraph: A High-Performance Graph Mining System with Pattern Decomposition
Jingji Chen, Xuehai Qian
Graph pattern mining (GPM) is an important application that identifies structures from graphs. Despite the recent progress, the performance gap between the state-of-the-art GPM sys…