4 papers
Latency-Quality Routing for Functionally Equivalent Tools in LLM Agents
Kexin Chu, Dawei Xiang, Wei Zhang
Tool-augmented LLM agents increasingly access the same tool type through multiple functionally equivalent providers, such as web-search APIs, retrievers, or LLM backends exposed be…
Dynamic Expert Quantization for Scalable Mixture-of-Experts Inference
Kexin Chu, Dawei Xiang, Zixu Shen +3
Mixture-of-Experts (MoE) has become a practical architecture for scaling LLM capacity while keeping per-token compute modest, but deploying MoE models on a single, memory-limited G…
ExpertFlow: Adaptive Expert Scheduling and Memory Coordination for Efficient MoE Inference
Zixu Shen, Kexin Chu, Yifan Zhang +3
The expansion of large language models is increasingly limited by the constrained memory capacity of modern GPUs. To mitigate this, Mixture-of-Experts (MoE) architectures activate…
MVVM: Deploy Your AI Agents-Securely, Efficiently, Everywhere
Yiwei Yang, Aibo Hu, Yusheng Zheng +6
The rise of AI agents powered by Large Language Models (LLMs) presents critical challenges: how to securely execute and migrate these agents across heterogeneous environments while…