5 papers
Scheduling LLM Inference with Uncertainty-Aware Output Length Predictions
Haoyu Zheng, Yongqiang Zhang, Fangcheng Fu +7
To schedule LLM inference, the \textit{shortest job first} (SJF) principle is favorable by prioritizing requests with short output lengths to avoid head-of-line (HOL) blocking. Exi…
Efficient Serving for Dynamic Agent Workflows with Prediction-based KV-Cache Management
Haoyu Zheng, Fangcheng Fu, Jia Wu +6
LLM-based workflows compose specialized agents to execute complex tasks, and these agents usually share substantial context, allowing KV-Cache reuse to save computation. Existing a…
AdaCultureSafe: Adaptive Cultural Safety Grounded by Cultural Knowledge in Large Language Models
Hankun Kang, Di Lin, Zhirong Liao +6
With the global proliferation of Large Language Models (LLMs), cultural safety, defined as the ability to generate respectful and appropriate responses across diverse cultures, bec…
Exploiting Text Semantics for Few and Zero Shot Node Classification on Text-attributed Graph
Yuxiang Wang, Xiao Yan, Shiyu Jin +6
Text-attributed graph (TAG) provides a text description for each graph node, and few- and zero-shot node classification on TAGs have many applications in fields such as academia an…
Hound: Hunting Supervision Signals for Few and Zero Shot Node Classification on Text-attributed Graph
Yuxiang Wang, Xiao Yan, Shiyu Jin +6
Text-attributed graph (TAG) is an important type of graph structured data with text descriptions for each node. Few- and zero-shot node classification on TAGs have many application…