3 citations · 6 across the 16 of their papers we have counts for
7 papers · 1 filter
Denoising Surface: Modeling and Predicting Inference Cost for Diffusion LLM Serving
Haoyu Zheng, Fangcheng Fu, Binhang Yuan +6
As diffusion large language models (dLLMs) become more capable, they are moving from research settings to real-world \textit{serving}, where request management (such as scheduling…
SiriusDeliver: Automating Data Warehouse Delivery at Tencent
Haining Xie, Xiaokai Zhou, Jiaming Yang +12
Enterprise data warehouses (DWs) support business-critical analytics, but warehouse task delivery remains a complicated production process involving context retrieval, workflow con…
MiCU: End-to-End Smart Home Command Understanding with Large Language Model
Haowei Han, Kexin Hu, Weiwei Cai +6
Command understanding systems in smart home ecosystems can automate device control and substantially improve user experience. However, while they perform well on precise utterances…
Scheduling LLM Inference with Uncertainty-Aware Output Length Predictions
Haoyu Zheng, Yongqiang Zhang, Fangcheng Fu +7
To schedule LLM inference, the \textit{shortest job first} (SJF) principle is favorable by prioritizing requests with short output lengths to avoid head-of-line (HOL) blocking. Exi…
Efficient Serving for Dynamic Agent Workflows with Prediction-based KV-Cache Management
Haoyu Zheng, Fangcheng Fu, Jia Wu +6
LLM-based workflows compose specialized agents to execute complex tasks, and these agents usually share substantial context, allowing KV-Cache reuse to save computation. Existing a…
UniVA: Unified Value Alignment for Generative Recommendation in Online Advertising at Tencent
Xinxun Zhang, Yuling Xiong, Yangru Huang +14
Generative Recommendation (GR) reformulates recommendation as next-token generation over item Semantic IDs (SIDs) and has shown promise in industrial applications. However, extendi…
MTServe: Efficient Serving for Generative Recommendation Models with Hierarchical Caches
Xin Wang, Chi Ma, Shaobin Chen +14
Generative recommendation (GR) offers superior modeling capabilities but suffers from prohibitive inference costs due to the repeated encoding of long user histories. While cross-r…