3 papers
cs.DC2026
Demystifying the Design Space and Best Practices for Heterogeneous LLM Inference and Serving
Zhixin Wang, Zhengbo Wang, Fangcheng Fu +15
Heterogeneous prefill-decode (PD) inference is now in production: prefill on cost-efficient or supply-available accelerators, decode on bandwidth-strong ones, and KV state crossing…
cs.DC2026
W4A16 Mixed-Precision Matrix Multiplication on Decoupled Architecture: Kernel Design and Memory Bottleneck Analysis for Ascend NPUs
Yuanhong He, Peiyu Niu, Jun Chen +2
As Large Language Models (LLMs) scale, weight-only quantization (W4A16: 4-bit weights, 16-bit activations) becomes critical for reducing memory footprint with minimal accuracy loss…
cs.DC2025
An Efficient, Reliable and Observable Collective Communication Library in Large-scale GPU Training Clusters
Mingjun Zhang, Xiaohe Hu, Menghao Zhang +21
Large-scale LLM training requires collective communication libraries to exchange data among distributed GPUs. As a company dedicated to building and operating large-scale GPU train…