Clairvoyant: Predictive Shortest-Job-First Admission for Serial LLM Inference

Aravind Sundaresan

Serial LLM inference backends process requests sequentially under First-Come-First-Served (FCFS) admission, causing Head-of-Line Blocking (HOLB) under mixed workloads: short factua…