5 papers
StrataCL: Fabric-Native Communication Library for Production Supernodes
Tiancheng Hu, Jin Qin, Yuzheng Wang +14
Modern distributed AI workloads run across hundreds of accelerators, making communication a major bottleneck. Existing communication libraries remain largely buffer-centric because…
UBEP: Re-architecting Expert Parallelism Communication Library for Production Superpods
Yipeng Liu, Chang Liu, Si Shen +16
The deployment of Mixture-of-Experts (MoE) models on production high-bandwidth superpods, such as NVIDIA's NVL72/576 and Huawei's CloudMatrix384, introduces critical challenges bey…
Rethinking Depth Pruning for Vision Transformers: A Heterogeneity-Aware Perspective
Zhenfeng Su, Kang Zhao, Han Bao +4
While prior studies have successfully compressed vision Transformers (ViTs) through various pruning techniques, most have concentrated on width pruning to achieve significant reduc…
Relay Buffer Independent Communication over Pooled HBM for Efficient MoE Inference on Ascend
Tianlun Hu, Tiancheng Hu, Shengsheng Litang +8
Mixture-of-Experts (MoE) inference requires large-scale token exchange across devices, making dispatch and combine major bottlenecks in both prefill and decode. Beyond network tran…
ENEC: A Lossless AI Model Compression Method Enabling Fast Inference on Ascend NPUs
Jinwu Yang, Jiaan Wu, Zedong Liu +17
The rapid scaling of Large Language Models presents significant challenges for their deployment and inference, particularly on resource-constrained specialized AI hardware accelera…