9 citations · 17 across the 8 of their papers we have counts for
8 papers
MegaScale-Omni: A Hyper-Scale, Workload-Resilient System for MultiModal LLM Training in Production
Chunyu Xue, Yangrui Chen, Jianyu Jiang +14
As the foundational component of versatile AI applications, training an multimodal large language model (MLLM) relies on multimodal datasets with dynamic modality mixture proportio…
Mesh-Attention: A New Communication-Efficient Distributed Attention with Improved Data Locality
Sirui Chen, Jingji Chen, Siqi Zhu +3
Distributed attention is essential for scaling large language models (LLMs) to long contexts, yet existing methods either have limited parallelism or incur high communication costs…
Virtual Width Networks
Seed, Baisheng Li, Banggu Wu +115
We introduce Virtual Width Networks (VWN), a framework that delivers the benefits of wider representations without incurring the quadratic cost of increasing the hidden size. VWN d…
Seed1.5-VL Technical Report
Dong Guo, Faming Wu, Feida Zhu +194
We present Seed1.5-VL, a vision-language foundation model designed to advance general-purpose multimodal understanding and reasoning. Seed1.5-VL is composed with a 532M-parameter v…
GNNPipe: Scaling Deep GNN Training with Pipelined Model Parallelism
Jingji Chen, Zhuoming Chen, Xuehai Qian
Communication is a key bottleneck for distributed graph neural network (GNN) training. This paper proposes GNNPipe, a new approach that scales the distributed full-graph deep GNN t…
Kudu: An Efficient and Scalable Distributed Graph Pattern Mining Engine
Jingji Chen, Xuehai Qian
This paper proposes Kudu, a distributed execution engine with a well-defined abstraction that can be integrated with existing single-machine graph pattern mining (GPM) systems to p…
IntersectX: An Efficient Accelerator for Graph Mining
Gengyu Rao, Jingji Chen, Jason Yik +1
Graph pattern mining applications try to find all embeddings that match specific patterns. Compared to the traditional graph computation, graph mining applications are computation-…
DwarvesGraph: A High-Performance Graph Mining System with Pattern Decomposition
Jingji Chen, Xuehai Qian
Graph pattern mining (GPM) is an important application that identifies structures from graphs. Despite the recent progress, the performance gap between the state-of-the-art GPM sys…