15 papers
SETA: Scaling Environments for Terminal Agents
Qijia Shen, Zhiqi Huang, Vamsidhar Kamanuru +19
Large language models (LLMs) are rapidly shifting toward agents that solve tasks through diverse interfaces, including web and graphical user interfaces (GUIs). Among these, the te…
Continuum: Efficient and Robust Multi-Turn LLM Agent Scheduling with KV Cache Time-to-Live
Hanchen Li, Runyuan He, Qiuyang Mang +7
KV cache management is essential for efficient LLM inference. To maximize utilization, existing inference engines evict finished requests' KV cache if new requests are waiting. Thi…
FrontierSmith: Synthesizing Open-Ended Coding Problems at Scale
Runyuan He, Qiuyang Mang, Shang Zhou +14
Many real-world coding challenges are open-ended and admit no known optimal solution. Yet, recent progress in LLM coding has focused on well-defined tasks such as feature implement…
Sieve: Dynamic Expert-Aware PIM Acceleration for Evolving Mixture-of-Experts Models
Jungwoo Kim, Rubens Lacouture, Genghan Zhang +5
Mixture-of-Experts (MoE) has become a dominant architecture for scaling large language models (LLMs). However, the execution characteristics of MoE inference are changing rapidly a…
Switchcraft: AI Model Router for Agentic Tool Calling
Sharad Agarwal, Pooria Namyar, Alec Wolman +3
Agentic AI systems that invoke external tools are powerful but costly, leading developers to default to large models and overspend inference budgets. Model routing can mitigate thi…
Combee: Scaling Prompt Learning for Self-Improving Language Model Agents
Hanchen Li, Runyuan He, Qizheng Zhang +11
Recent advances in prompt learning allow large language model agents to acquire task-relevant knowledge from inference-time context without parameter changes. For example, existing…