6 papers
NanoCP: Request-Level Dynamic Context Parallelism for Data-Expert Parallel Decoding
Jiefei Chen, Binbin Lin, Jinming Ma +9
Modern serving systems for Mixture-of-Experts (MoE) models adopt hybrid data-expert parallelism: expert parallelism (EP) shards experts across GPUs to scale capacity, while data pa…
VFA: Relieving Vector Operations in Flash Attention with Global Maximum Pre-computation
Yupeng Sun, Yanzhao Li, Zhiqiang Zou +5
FlashAttention-style online softmax enables exact attention computation with linear memory by streaming score tiles through on-chip memory and maintaining a running maximum and nor…
OSC: Hardware Efficient W4A4 Quantization via Outlier Separation in Channel Dimension
Zhiyuan Zhang, Yanzhao Li, Zhiqiang Zou +4
While 4-bit quantization is essential for high-throughput deployment of Large Language Models, activation outliers often lead to significant accuracy degradation due to the restric…
TC-GS: A Faster Gaussian Splatting Module Utilizing Tensor Cores
Zimu Liao, Jifeng Ding, Siwei Cui +7
3D Gaussian Splatting (3DGS) renders pixels by rasterizing Gaussian primitives, where conditional alpha-blending dominates the computational cost in the rendering pipeline. This pa…
TMA-Adaptive FP8 Grouped GEMM: Eliminating Padding Requirements in Low-Precision Training and Inference on Hopper
Zhongling Su, Rong Fu, Weihan Cao +4
Current FP8 grouped GEMM implementations require padding each group to a fixed alignment (e.g., 128), incurring memory and computational overhead. We propose \textit{TMA-Adaptive F…
H2:Towards Efficient Large-Scale LLM Training on Hyper-Heterogeneous Cluster over 1,000 Chips
Ding Tang, Jiecheng Zhou, Jiakai Hu +5
Recent advancements in large language models (LLMs) necessitate extensive computational resources, prompting the use of diverse hardware accelerators from multiple vendors. However…