6 papers
Idleness is Relative: Exploiting Tool-Call Idle Windows for Offloading in Agentic Systems with MORI
Tian Xia, Hanchen Li, Zhifei Li +5
Modern LLM serving systems increasingly host agentic workloads, whose sessions issue tens of model invocations interleaved with tool calls, accumulating KV cache that can be reused…
EvoX: Meta-Evolution for Automated Discovery
Shu Liu, Shubham Agarwal, Monishwaran Maheswaran +14
Recent work such as AlphaEvolve has shown that combining LLM-driven optimization with evolutionary search can effectively improve programs, prompts, and algorithms across domains.…
SkyNomad: On Using Multi-Region Spot Instances to Minimize AI Batch Job Cost
Zhifei Li, Tian Xia, Ziming Mao +9
AI batch jobs such as model training, inference pipelines, and data analytics require substantial GPU resources and often need to finish before a deadline. Spot instances offer 3-1…
LEANN: A Low-Storage Vector Index
Yichuan Wang, Zhifei Li, Shu Liu +10
Embedding-based vector search underpins many important applications, such as recommendation and retrieval-augmented generation (RAG). It relies on vector indices to enable efficien…
SkyWalker: A Locality-Aware Cross-Region Load Balancer for LLM Inference
Tian Xia, Ziming Mao, Jamison Kerney +5
Serving Large Language Models (LLMs) efficiently in multi-region setups remains a challenge. Due to cost and GPU availability concerns, providers typically deploy LLMs in multiple…
Barbarians at the Gate: How AI is Upending Systems Research
Audrey Cheng, Shu Liu, Melissa Pan +14
Artificial Intelligence (AI) is starting to transform the research process as we know it by automating the discovery of new solutions. Given a task, the typical AI-driven approach…