From the 1 of 6 linked papers with an AI index.
6 papers
StrataCL: Fabric-Native Communication Library for Production Supernodes
Tiancheng Hu, Jin Qin, Yuzheng Wang +14
StrataCL is a communication library for large‑scale AI workloads on supernode clusters that removes redundant data copies by enabling direct user‑buffer communication and uses NPU‑…
Decode-Time Grammars: Constrained LLM Generation over a Refinement Order of Grammar Fragments
Shuoming Zhang, Ruiyuan Xu, Haofeng Li +7
Large language models now write a growing share of the world's code, increasingly inside agents and serving systems that compile, execute, or dispatch generated code without line-b…
Tessera: Unlocking Heterogeneous GPUs through Kernel-Granularity Disaggregation
Tiancheng Hu, Jin Qin, Zheng Wang +10
Disaggregation maps parts of an AI workload to different types of GPUs, offering a path to utilize modern heterogeneous GPU clusters. However, existing solutions operate at a coars…
Demystifying Numerical Instability in LLM Inference: Achieving Reproducible Inference for Mission-Critical Tasks with HEAL
Zhenting Zhu, Lucas Thai, Shan Yu +5
As Large Language Models (LLMs) deploy into mission-critical domains (e.g., finance, medicine, and law), output reproducibility has become a strict system requirement. While practi…
Hummingbird: SLO-Oriented GPU Preemption at Microsecond-scale
Tiancheng Hu, Chenxi Wang, Ting Cao +9
Existing GPU-sharing techniques, including spatial and temporal sharing, aim to improve utilization but face challenges in simultaneously ensuring SLO adherence and maximizing effi…
MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Pool
Cunchen Hu, Heyang Huang, Junhao Hu +8
Large language model (LLM) serving has transformed from stateless to stateful systems, utilizing techniques like context caching and disaggregated inference. These optimizations ex…