ThunderAgent: A Simple, Fast and Program-Aware Agentic Inference System

Hao Kang, Ziyang Li, Weili Xu +7

Large language models(LLMs) are now used to power complex multi-turn agentic workflows. Existing systems run agentic inference by loosely assembling isolated components: an LLM inf…