From the 1 of 5 linked papers with an AI index.
5 papers
StrataCL: Fabric-Native Communication Library for Production Supernodes
Tiancheng Hu, Jin Qin, Yuzheng Wang +14
StrataCL is a communication library for large‑scale AI workloads on supernode clusters that removes redundant data copies by enabling direct user‑buffer communication and uses NPU‑…
UBEP: Re-architecting Expert Parallelism Communication Library for Production Superpods
Yipeng Liu, Chang Liu, Si Shen +16
The deployment of Mixture-of-Experts (MoE) models on production high-bandwidth superpods, such as NVIDIA's NVL72/576 and Huawei's CloudMatrix384, introduces critical challenges bey…
Rethinking Depth Pruning for Vision Transformers: A Heterogeneity-Aware Perspective
Zhenfeng Su, Kang Zhao, Han Bao +4
While prior studies have successfully compressed vision Transformers (ViTs) through various pruning techniques, most have concentrated on width pruning to achieve significant reduc…
Relay Buffer Independent Communication over Pooled HBM for Efficient MoE Inference on Ascend
Tianlun Hu, Tiancheng Hu, Shengsheng Litang +8
Mixture-of-Experts (MoE) inference requires large-scale token exchange across devices, making dispatch and combine major bottlenecks in both prefill and decode. Beyond network tran…
ENEC: A Lossless AI Model Compression Method Enabling Fast Inference on Ascend NPUs
Jinwu Yang, Jiaan Wu, Zedong Liu +17
The rapid scaling of Large Language Models presents significant challenges for their deployment and inference, particularly on resource-constrained specialized AI hardware accelera…