4 papers
TOPAS: Workflow-Aware Prefix-State Scheduling for Multi-Agent LLM Serving
Hongqiu Ni, Han Tian, Chi Zhang +2
Prefix caching introduces a fundamental tradeoff in multi-agent large language model (LLM) serving: retaining a long system-prompt key-value (KV) cache for an agent accelerates fut…
Embedding Samples Dispatching for Recommendation Model Training in Edge Environments
Guopeng Li, Haisheng Tan, Chi Zhang +5
Training deep learning recommendation models (DLRMs) on edge workers brings several benefits, particularly in terms of data privacy protection, low latency and personalization. How…
A Plan Reuse Mechanism for LLM-Driven Agent
Guopeng Li, Ruiqi Wu, Haisheng Tan
Integrating large language models (LLMs) into personal assistants, like Xiao Ai and Blue Heart V, effectively enhances their ability to interact with humans, solve complex tasks, a…
Astraea: A State-Aware Scheduling Engine for LLM-Powered Agents
Hongqiu Ni, Jiabao Zhang, Guopeng Li +4
Large Language Models (LLMs) are increasingly being deployed as intelligent agents. Their multi-stage workflows, which alternate between local computation and calls to external net…