activity
20212026
most citedSpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification

101 citations · 158 across the 12 of their papers we have counts for

collaborators
Showing cs.DCShow all

2 papers · 1 filter

cs.DC2026

Event Tensor: A Unified Abstraction for Compiling Dynamic Megakernel

Hongyi Jin, Bohan Hou, Guanjie Wang +18

Modern GPU workloads, especially large language model (LLM) inference, suffer from kernel launch overheads and coarse synchronization that limit inter-kernel parallelism. Recent me…

cs.DC2024

FlexLLM: Token-Level Co-Serving of LLM Inference and Finetuning with SLO Guarantees

Gabriele Oliaro, Xupeng Miao, Xinhao Cheng +9

Finetuning large language models (LLMs) is essential for task adaptation, yet today's serving stacks isolate inference and finetuning on separate GPU clusters -- wasting resources…