Duration Aware Scheduling for ASR Serving Under Workload Drift

Darshan Makwana, Yash Jogi, Harsh Kotta +1

Scheduling policies in large-scale Automatic Speech Recognition (ASR) serving pipelines play a key role in determining end-to-end (E2E) latency. Yet, widely used serving engines re…