4 papers
Towards Multi-Model LLM Schedulers: Empirical Insights into Offloading and Preemption
Mert Yildiz, Pietro Spadaccino, Alexey Rolich +2
Modern deployments of Large Language Models (LLMs) increasingly require serving multiple models with diverse architectures, sizes, and specialization on shared, heterogeneous hardw…
"Two-Stagification": Job Dispatching in Large-Scale Clusters via a Two-Stage Architecture
Mert Yildiz, Alexey Rolich, Andrea Baiocchi
A continuing effort is devoted to devising effective dispatching policies for clusters of First Come First Served servers. Although the optimal solution for dispatchers aware of bo…
Dispatching Odyssey: Exploring Performance in Computing Clusters under Real-world Workloads
Mert Yildiz, Alexey Rolich, Andrea Baiocchi
Recent workload measurements in Google data centers provide an opportunity to challenge existing models and, more broadly, to enhance the understanding of dispatching policies in c…
The Merit of Simple Policies: Buying Performance With Parallelism and System Architecture
Mert Yildiz, Alexey Rolich, Andrea Baiocchi
While scheduling and dispatching of computational workloads is a well-investigated subject, only recently has Google provided publicly a vast high-resolution measurement dataset of…