3 papers
cs.PL2025
veScale: Consistent and Efficient Tensor Programming with Eager-Mode SPMD
Youjie Li, Cheng Wan, Zhiqi Lin +10
Large Language Models (LLMs) have scaled rapidly in size and complexity, requiring increasingly intricate parallelism for distributed training, such as 3D parallelism. This sophist…
cs.DC2025
TileLink: Generating Efficient Compute-Communication Overlapping Kernels using Tile-Centric Primitives
Size Zheng, Jin Fang, Xuegui Zheng +9
Large deep learning models have achieved state-of-the-art performance in a wide range of tasks. These models often necessitate distributed systems for efficient training and infere…
cs.DC2025
Triton-distributed: Programming Overlapping Kernels on Distributed AI Systems with the Triton Compiler
Size Zheng, Wenlei Bao, Qi Hou +19
In this report, we propose Triton-distributed, an extension of existing Triton compiler, to overcome the programming challenges in distributed AI systems. Triton-distributed is the…