4 papers
CommBench: Can LLMs Write Correct and Efficient GPU Communication Code?
Shuang Ma, Yuyi Li, Yihan Zhang +12
Training and serving large language models (LLMs) rely heavily on high-performance GPU communication, yet implementing efficient GPU communication primitives requires deep expertis…
UCCL-EP: Portable Expert-Parallel Communication
Ziming Mao, Yihan Zhang, Chihan Cui +9
Mixture-of-Experts (MoE) workloads rely on expert parallelism (EP) to achieve high GPU efficiency. State-of-the-art EP communication systems such as DeepEP demonstrate strong perfo…
Exploring Topological Bias in Heterogeneous Graph Neural Networks
Yihan Zhang
Graph Neural Networks (GNNs) are characterized by their capacity of processing graph-structured data. However, due to the sparsity of labels under semi-supervised learning, they ha…
Tight Bounds on List-Decodable and List-Recoverable Zero-Rate Codes
Nicolas Resch, Chen Yuan, Yihan Zhang
In this work, we consider the list-decodability and list-recoverability of codes in the zero-rate regime. Briefly, a code is -list-recover…