LLMSched: Uncertainty-Aware Workload Scheduling for Compound LLM Applications

Botao Zhu, Chen Chen, Xiaoyi Fan +1

Developing compound Large Language Model (LLM) applications is becoming an increasingly prevalent approach to solving real-world problems. In these applications, an LLM collaborate…