681 citations · 973 across the 51 of their papers we have counts for
39 papers · 1 filter
DRLM: Deep Reinforcement Learning-Based LLM Query Orchestration in Edge Environments
Reza Farahani, Zoha Azimi Ourimi, Mario Colosi +3
Large language model (LLM) services increasingly process heterogeneous queries with diverse latency, accuracy, and resource requirements. While edge deployment reduces response tim…
LipCache: A Local Inference Proxy with Certified Caching for Edge Image Classification Service
Zhengzhe Xiang, Yinlin Chen, Fuli Ying +3
As edge-side vision services continue to expand toward low-latency, high-throughput scenarios, reducing the inference cost of vision models without sacrificing reliability has beco…
A Taxonomy of Performance Metrics for the Distributed Computing Continuum
Praveen Kumar Donta, Boris Sedlak, Alfreds Lapkovskis +6
Performance evaluation is essential for understanding, comparing, and improving computing systems, including Distributed Computing Continuum Systems (DCCS). In recent years, comput…
LMEdge: QoS-Aware LLM Inference Orchestration on Edge Clusters
Reza Farahani, Zoha Azimi, Mario Colosi +1
Large language model (LLM) services increasingly operate on edge infrastructure, enabling low-latency and privacy-preserving AI services. However, efficiently serving LLM requests…
Incentives and Evidence in Learned Service Orchestration
Syed Izhan Khilji, Alireza Furutanpey, Schahram Dustdar
Reinforcement learning for service orchestration has been the subject of sustained research for over a decade, yet it is not used in production at scale. The usual explanation is t…
Fair Comparison of Scheduling Algorithms on Heterogeneous Edge Clusters: A Continuous Adaptive Benchmark
Zihang Wang, Boris Sedlak, Juan Luis Herrera +1
Modern Artificial Intelligence (AI) workloads deployed across the heterogeneous tiers of an edge--cloud continuum must satisfy multi-dimensional Service Level Objectives (SLOs) ove…