25 citations · 25 across the 13 of their papers we have counts for
15 papers
vSkipper: Translating Dynamic Layer Skipping into LLM Serving Gains
Wei Da, Yavuz Ferhatosmanoglu, Evangelia Kalyvianaki
Dynamic layer skipping reduces LLM computation by allowing each token to execute only a subset of the model's layers. However, existing skippers rely on specialized generation loop…
RouteBalance: Fused Model Routing and Load Balancing for Heterogeneous LLM Serving
Wei Da, Evangelia Kalyvianaki
Current heterogeneous LLM serving clusters decouple request scheduling into two independent tiers: learned model routers select a model based on the prompt without visibility into…
LLM-Emu: Native Runtime Emulation of LLM Inference via Profile-Driven Sampling
Wei Da, Evangelia Kalyvianaki
Realistic evaluation of LLM serving systems requires online workloads, dynamic arrivals, queueing, and the serving engine's local scheduling for execution batching, but running suc…
Dodoor: Efficient Randomized Decentralized Scheduling with Load Caching for Heterogeneous Tasks and Clusters
Wei Da, Evangelia Kalyvianaki
This paper presents Dodoor, a randomized decentralized scheduler for heterogeneous clusters. Dodoor removes hot-path probing via batched cache refreshes and introduces a heterogene…
Mitigating context switching in densely packed Linux clusters with Latency-Aware Group Scheduling
Al Amjad Tawfiq Isstaif, Evangelia Kalyvianaki, Richard Mortier
Cluster orchestrators such as Kubernetes depend on accurate estimates of node capacity and job requirements. Inaccuracies in either lead to poor placement decisions and degraded cl…
Astrolabe: Balancing Load in LLM Serving with Randomized Prediction-Guided Scheduling
Wei Da, Evangelia Kalyvianaki
This paper presents Astrolabe, a randomized prediction-guided scheduler for one-shot request dispatch in multi-instance large language model (LLM) serving. Astrolabe improves load…
Average Consensus over Directed Networks in Open Multi-Agent Systems with Acknowledgement Feedback
Evagoras Makridis, Andreas Grammenos, Gabriele Oliva +3
In this paper, we address the distributed average consensus problem over directed networks in open multi-agent systems (OMAS), where the stability of the network is disrupted by fr…
Distributed Optimization for Quadratic Cost Functions over Large-Scale Networks with Quantized Communication and Finite-Time Convergence
Apostolos I. Rikos, Andreas Grammenos, Evangelia Kalyvianaki +3
We propose two distributed iterative algorithms that can be used to solve, in finite time, the distributed optimization problem over quadratic local cost functions in large-scale n…
Distributed CPU Scheduling Subject to Nonlinear Constraints
Mohammadreza Doostmohammadian, Alireza Aghasi, Apostolos I. Rikos +5
This paper considers a network of collaborating agents for local resource allocation subject to nonlinear model constraints. In many applications, it is required (or desirable) tha…
Pronto: Federated Task Scheduling
Andreas Grammenos, Evangelia Kalyvianaki, Peter Pietzuch
We present a federated, asynchronous, memory-limited algorithm for online task scheduling across large-scale networks of hundreds of workers. This is achieved through recent advanc…
An Asynchronous Approximate Distributed Alternating Direction Method of Multipliers in Digraphs
Wei Jiang, Andreas Grammenos, Evangelia Kalyvianaki +1
In this work, we consider the asynchronous distributed optimization problem in which each node has its own convex cost function and can communicate directly only with its neighbors…
Optimal CPU Scheduling in Data Centers via a Finite-Time Distributed Quantized Coordination Mechanism
Apostolos I. Rikos, Andreas Grammenos, Evangelia Kalyvianaki +3
In this paper we analyze the problem of optimal task scheduling for data centers. Given the available resources and tasks, we propose a fast distributed iterative algorithm which o…
CPU Scheduling in Data Centers Using Asynchronous Finite-Time Distributed Coordination Mechanisms
Andreas Grammenos, Themistoklis Charalambous, Evangelia Kalyvianaki
We propose an asynchronous iterative scheme that allows a set of interconnected nodes to distributively reach an agreement within a pre-specified bound in a finite number of steps.…
Robust Dynamic CPU Resource Provisioning in Virtualized Servers
Evagoras Makridis, Kyriakos Deliparaschos, Evangelia Kalyvianaki +2
We present robust dynamic resource allocation mechanisms to allocate application resources meeting Service Level Objectives (SLOs) agreed between cloud providers and customers. In…
Self-Awareness of Cloud Applications
Alexandru Iosup, Xiaoyun Zhu, Arif Merchant +6
Cloud applications today deliver an increasingly larger portion of the Information and Communication Technology (ICT) services. To address the scale, growth, and reliability of clo…