248 citations · 269 across the 27 of their papers we have counts for
7 papers · 1 filter
DySCo: Dynamic Sharding for Collaborative Edge-Cloud LLM Inference with Depth-Synchronized Batching
Jingpo Xu, Paul Joe Maliakel, Ivona Brandic +1
Pervasive intelligent applications are increasingly deployed on mobile and Internet of Things (IoT) edge devices. Consequently, Large Language Models (LLMs) are increasingly used t…
A Technique for Load Shifting Low-latency Applications in Multi-Region Renewables Harvesting via SMT Core Pooling
Tharindu B. Hewage, Shashikant Ilager, Maria A. Rodriguez +1
Load shifting across geographic regions to chase intermittent renewable energy availability is commonly used in reducing cloud infrastructure carbon footprint. However, it often om…
Carbon-aware Resource Management for Latency-Sensitive Cloud Computing Environments: A Taxonomy and Future Directions
Tharindu B. Hewage, Shashikant Ilager, Maria Rodriguez Read +1
The proliferation of cloud-based latency-sensitive workloads, such as autonomous driving, real-time healthcare services, industrial automation, immersive interactive services, and…
EcoKube: Simulating Carbon-Aware Scheduling Policies in Heterogeneous Edge-Cloud Environments
Gonçalo Ferreira, Shashikant Ilager
Energy demand from cloud and edge computing is rising rapidly, with AI workloads further intensifying electricity use and associated carbon emissions. In hybrid edge-cloud settings…
Leveraging LLMs for Structured Information Extraction and Analysis from Cloud Incident Reports (Work In Progress Paper)
Xiaoyu Chu, Shashikant Ilager, Yizhen Zang +2
Incident management is essential to maintain the reliability and availability of cloud computing services. Cloud vendors typically disclose incident reports to the public, summariz…
GreenServ: Energy-Efficient Context-Aware Dynamic Routing for Multi-Model LLM Inference
Thomas Ziller, Shashikant Ilager, Alessandro Tundo +3
Large language models (LLMs) demonstrate remarkable capabilities, but their broad deployment is limited by significant computational resource demands, particularly energy consumpti…
ARKV: Adaptive and Resource-Efficient KV Cache Management under Limited Memory Budget for Long-Context Inference in LLMs
Jianlong Lei, Shashikant Ilager
Large Language Models (LLMs) are increasingly deployed in scenarios demanding ultra-long context reasoning, such as agentic workflows and deep research understanding. However, long…