FATE: Future-State-Aware Scheduling for Heterogeneous LLM Workflows

Zirui Huang, Yi-Xiang Hu, Feng Wu +1

Large language model (LLM) applications are increasingly executed as heterogeneous multi-stage workflows rather than isolated inference calls. In these workflow directed acyclic gr…