57 citations · 75 across the 26 of their papers we have counts for
11 papers · 1 filter
AReaL-Hex: Accommodating Asynchronous RL Training over Heterogeneous GPUs
Ran Yan, Youhe Jiang, Tianyuan Wu +7
Maximizing training throughput and cost-efficiency of RL for LLMs is essential to democratize this advanced technique. One promising but challenging approach is to deploy such a co…
Parallax: Efficient LLM Inference Service over Decentralized Environment
Chris Tong, Youhe Jiang, Gufeng Chen +6
Deploying a large language model (LLM) inference service remains costly because centralized serving depends on specialized GPU clusters and high-bandwidth interconnects in datacent…
Efficient Pre-Training of LLMs via Topology-Aware Communication Alignment on More Than 9600 GPUs
Guoliang He, Youhe Jiang, Wencong Xiao +8
The scaling law for large language models (LLMs) depicts that the path towards machine intelligence necessitates training at large scale. Thus, companies continuously build large-s…
FSA: An Alternative Efficient Implementation of Native Sparse Attention Kernel
Ran Yan, Youhe Jiang, Zhuoming Chen +3
Recent advances in sparse attention mechanisms have demonstrated strong potential for reducing the computational cost of long-context training and inference in large language model…
LMDeploy Accelerates Mixed-Precision LLM Inference with TurboMind
Li Zhang, Youhe Jiang, Guoliang He +6
Mixed-precision inference techniques reduce the memory and computational demands of Large Language Models (LLMs) by applying hybrid precision formats to model weights, activations,…
Cascadia: An Efficient Cascade Serving System for Large Language Models
Youhe Jiang, Fangcheng Fu, Wanru Zhao +4
Recent advances in large language models (LLMs) have intensified the need to deliver both rapid responses and high-quality outputs. More powerful models yield better results but in…
Thinking Short and Right Over Thinking Long: Serving LLM Reasoning Efficiently and Accurately
Yuhang Wang, Youhe Jiang, Bin Cui +1
Recent advances in test-time scaling suggest that Large Language Models (LLMs) can gain better capabilities by generating Chain-of-Thought reasoning (analogous to human thinking) t…
HEXGEN-FLOW: Optimizing LLM Inference Request Scheduling for Agentic Text-to-SQL
You Peng, Youhe Jiang, Wenqi Jiang +2
Recent advances in agentic large language models (LLMs) have substantially improved Text-to-SQL, enabling users without database expertise to query databases intuitively. However,…
HexGen-2: Disaggregated Generative Inference of LLMs in Heterogeneous Environment
Youhe Jiang, Ran Yan, Binhang Yuan
Disaggregating the prefill and decoding phases represents an effective new paradigm for generative inference of large language models (LLM), which eliminates prefill-decoding inter…
ThunderServe: High-performance and Cost-efficient LLM Serving in Cloud Environments
Youhe Jiang, Fangcheng Fu, Xiaozhe Yao +4
Recent developments in large language models (LLMs) have demonstrated their remarkable proficiency in a range of tasks. Compared to in-house homogeneous GPU clusters, deploying LLM…
Demystifying Cost-Efficiency in LLM Serving over Heterogeneous GPUs
Youhe Jiang, Fangcheng Fu, Xiaozhe Yao +6
Recent advancements in Large Language Models (LLMs) have led to increasingly diverse requests, accompanied with varying resource (compute and memory) demands to serve them. However…