3 citations · 6 across the 16 of their papers we have counts for
8 papers · 1 filter
Denoising Surface: Modeling and Predicting Inference Cost for Diffusion LLM Serving
Haoyu Zheng, Fangcheng Fu, Binhang Yuan +6
As diffusion large language models (dLLMs) become more capable, they are moving from research settings to real-world \textit{serving}, where request management (such as scheduling…
Scheduling LLM Inference with Uncertainty-Aware Output Length Predictions
Haoyu Zheng, Yongqiang Zhang, Fangcheng Fu +7
To schedule LLM inference, the \textit{shortest job first} (SJF) principle is favorable by prioritizing requests with short output lengths to avoid head-of-line (HOL) blocking. Exi…
Efficient Serving for Dynamic Agent Workflows with Prediction-based KV-Cache Management
Haoyu Zheng, Fangcheng Fu, Jia Wu +6
LLM-based workflows compose specialized agents to execute complex tasks, and these agents usually share substantial context, allowing KV-Cache reuse to save computation. Existing a…
MTServe: Efficient Serving for Generative Recommendation Models with Hierarchical Caches
Xin Wang, Chi Ma, Shaobin Chen +14
Generative recommendation (GR) offers superior modeling capabilities but suffers from prohibitive inference costs due to the repeated encoding of long user histories. While cross-r…
RetroInfer: A Vector Storage Engine for Scalable Long-Context LLM Inference
Yaoqi Chen, Jinkai Zhang, Baotong Lu +16
Recent large language models (LLMs) are rapidly extending their context windows, yet inference throughput lags due to increasing GPU memory and bandwidth demands. This is because t…
Retrofitting Temporal Graph Neural Networks with Transformer
Qiang Huang, Xiao Yan, Xin Wang +5
Temporal graph neural networks (TGNNs) outperform regular GNNs by incorporating time information into graph-based operations. However, TGNNs adopt specialized models (e.g., TGN, TG…
TreeCSS: An Efficient Framework for Vertical Federated Learning
Qinbo Zhang, Xiao Yan, Yukai Ding +4
Vertical federated learning (VFL) considers the case that the features of data samples are partitioned over different participants. VFL consists of two main steps, i.e., identify t…
Generative and Contrastive Paradigms Are Complementary for Graph Self-Supervised Learning
Yuxiang Wang, Xiao Yan, Chuang Hu +5
For graph self-supervised learning (GSSL), masked autoencoder (MAE) follows the generative paradigm and learns to reconstruct masked graph edges or node features. Contrastive Learn…