6 citations · 13 across the 10 of their papers we have counts for
10 papers
StrataCL: Fabric-Native Communication Library for Production Supernodes
Tiancheng Hu, Jin Qin, Yuzheng Wang +14
Modern distributed AI workloads run across hundreds of accelerators, making communication a major bottleneck. Existing communication libraries remain largely buffer-centric because…
Decode-Time Grammars: Constrained LLM Generation over a Refinement Order of Grammar Fragments
Shuoming Zhang, Ruiyuan Xu, Haofeng Li +7
Large language models now write a growing share of the world's code, increasingly inside agents and serving systems that compile, execute, or dispatch generated code without line-b…
Demystifying Numerical Instability in LLM Inference: Achieving Reproducible Inference for Mission-Critical Tasks with HEAL
Zhenting Zhu, Lucas Thai, Shan Yu +5
As Large Language Models (LLMs) deploy into mission-critical domains (e.g., finance, medicine, and law), output reproducibility has become a strict system requirement. While practi…
Tessera: Unlocking Heterogeneous GPUs through Kernel-Granularity Disaggregation
Tiancheng Hu, Jin Qin, Zheng Wang +10
Disaggregation maps parts of an AI workload to different types of GPUs, offering a path to utilize modern heterogeneous GPU clusters. However, existing solutions operate at a coars…
Hummingbird: SLO-Oriented GPU Preemption at Microsecond-scale
Tiancheng Hu, Chenxi Wang, Ting Cao +9
Existing GPU-sharing techniques, including spatial and temporal sharing, aim to improve utilization but face challenges in simultaneously ensuring SLO adherence and maximizing effi…
A Tale of Two Paths: Toward a Hybrid Data Plane for Efficient Far-Memory Applications
Lei Chen, Shi Liu, Chenxi Wang +9
With rapid advances in network hardware, far memory has gained a great deal of traction due to its ability to break the memory capacity wall. Existing far memory systems fall into…
MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Pool
Cunchen Hu, Heyang Huang, Junhao Hu +8
Large language model (LLM) serving has transformed from stateless to stateful systems, utilizing techniques like context caching and disaggregated inference. These optimizations ex…
Inference without Interference: Disaggregate LLM Inference for Mixed Downstream Workloads
Cunchen Hu, Heyang Huang, Liangliang Xu +9
Transformer-based large language model (LLM) inference serving is now the backbone of many cloud services. LLM inference consists of a prefill phase and a decode phase. However, ex…
Canvas: Isolated and Adaptive Swapping for Multi-Applications on Remote Memory
Chenxi Wang, Yifan Qiao, Haoran Ma +6
Remote memory techniques for datacenter applications have recently gained a great deal of popularity. Existing remote memory techniques focus on the efficiency of a single applicat…
SpecBox: A Label-Based Transparent Speculation Scheme Against Transient Execution Attacks
Bowen Tang, Chenggang Wu, Zhe Wang +6
Speculative execution techniques have been a cornerstone of modern processors to improve instruction-level parallelism. However, recent studies showed that this kind of techniques…