25 citations · 25 across the 13 of their papers we have counts for
8 papers · 1 filter
vSkipper: Translating Dynamic Layer Skipping into LLM Serving Gains
Wei Da, Yavuz Ferhatosmanoglu, Evangelia Kalyvianaki
Dynamic layer skipping reduces LLM computation by allowing each token to execute only a subset of the model's layers. However, existing skippers rely on specialized generation loop…
RouteBalance: Fused Model Routing and Load Balancing for Heterogeneous LLM Serving
Wei Da, Evangelia Kalyvianaki
Current heterogeneous LLM serving clusters decouple request scheduling into two independent tiers: learned model routers select a model based on the prompt without visibility into…
LLM-Emu: Native Runtime Emulation of LLM Inference via Profile-Driven Sampling
Wei Da, Evangelia Kalyvianaki
Realistic evaluation of LLM serving systems requires online workloads, dynamic arrivals, queueing, and the serving engine's local scheduling for execution batching, but running suc…
Dodoor: Efficient Randomized Decentralized Scheduling with Load Caching for Heterogeneous Tasks and Clusters
Wei Da, Evangelia Kalyvianaki
This paper presents Dodoor, a randomized decentralized scheduler for heterogeneous clusters. Dodoor removes hot-path probing via batched cache refreshes and introduces a heterogene…
Astrolabe: Balancing Load in LLM Serving with Randomized Prediction-Guided Scheduling
Wei Da, Evangelia Kalyvianaki
This paper presents Astrolabe, a randomized prediction-guided scheduler for one-shot request dispatch in multi-instance large language model (LLM) serving. Astrolabe improves load…
Pronto: Federated Task Scheduling
Andreas Grammenos, Evangelia Kalyvianaki, Peter Pietzuch
We present a federated, asynchronous, memory-limited algorithm for online task scheduling across large-scale networks of hundreds of workers. This is achieved through recent advanc…
Optimal CPU Scheduling in Data Centers via a Finite-Time Distributed Quantized Coordination Mechanism
Apostolos I. Rikos, Andreas Grammenos, Evangelia Kalyvianaki +3
In this paper we analyze the problem of optimal task scheduling for data centers. Given the available resources and tasks, we propose a fast distributed iterative algorithm which o…
CPU Scheduling in Data Centers Using Asynchronous Finite-Time Distributed Coordination Mechanisms
Andreas Grammenos, Themistoklis Charalambous, Evangelia Kalyvianaki
We propose an asynchronous iterative scheme that allows a set of interconnected nodes to distributively reach an agreement within a pre-specified bound in a finite number of steps.…