15 papers
EvoCache: Accelerating Browser Large Language Model Delivery with Public Cache Anchors
Zhiyu Wang, Rajkumar Buyya
Large Language Models (LLMs) can run directly in a browser, but first use typically requires downloading model files that are hundreds of MiB or even several GiB in size. The same…
Darpan: A Digital Twin Framework for the Next-Generation Computing Continuum
Zhiyu Wang, Rajkumar Buyya
Computing-continuum applications distribute work across devices, edge systems, fog resources, and clouds. While a placement, scheduling, or recovery decision is being made, resourc…
Kafila: Serving Large Language Models on a Trusted Set of Heterogeneous Commodity Machines
Murtaza Rangwala, Richard O. Sinnott, Rajkumar Buyya
Between them, the members of a research group or a circle of friends own several consumer computers, none large enough to run a capable large language model. Existing systems pool…
Rollout Efficiency in Reinforcement Learning for Reasoning Large Language Models: A Taxonomy and Future Directions
Niloofar Gholipour, Marcos Assuncao, Gursimran Singh +8
Reasoning-oriented reinforcement learning enables large language models to solve mathematical, coding, and other multi-step tasks, but shifts a substantial portion of the training…
Agentic Autoscaling through Worker-Pool Orchestration for LLM-driven Text Classification in Cloud Computing Environments
Bablu Kumar, Anshul Verma, Rajkumar Buyya
The growing adoption of large language model (LLM)-based systems for large-scale text processing has created a critical need for dynamic autoscaling to manage high-latency, bursty,…
Stability-Aware Proactive Autoscaling Using a Double Deep Q-Network in Cloud Computing Environments
Bablu Kumar, Anshul Verma, Rajkumar Buyya
Dynamic workloads and latency-sensitive applications require efficient autoscaling in cloud computing environments. However, most existing approaches rely on reactive mechanisms ba…
CLASP: Chained-Request-Aware Scaling and Operator Placement for Serverless Stream Processing
Tianyu Qi, Maria A. Rodriguez, Rajkumar Buyya
Stateful serverless (Function-as-a-Service) environments, whose workers host state servers, are increasingly used for stream processing. A stream application is a pipeline of opera…
AgentR A Stateful and Recovery-Aware Software Architecture for LLM-based Auditable Workflows
Riya Samanta, Bidyut Saha, Soumya Kanti Ghosh +1
Modern LLM-based applications increasingly require multi- stage execution, persistent intermediate state, retry seman- tics, and auditable usage accounting. However, many LLM appli…
How Much Reconstruction Does Quantum Machine Learning Need? Late Fusion of Uncoupled Quantum Subcircuits
Prabhjot Singh, Adel N. Toosi, Rajkumar Buyya
Circuit cutting lets a large quantum neural network (QNN) run as independent subcircuits on small devices, but rebuilding its outputs by quasiprobability reconstruction carries a c…
FedCARE: A Multi-Objective Personalised Federated Learning Framework for Smart Healthcare
Rojalini Tripathy, Padmalochan Bera, Shreya Ghosh +1
Federated Learning (FL) enables collaborative model training across distributed healthcare institutions without centralising sensitive patient data. However, real-world healthcare…
Trust-Aware Topology Learning for Dynamic Decentralized Federated Learning under Adversaries
Shubham Vaishnav, Murtaza Rangwala, Ali Beikmohammadi +2
In dynamic mobile decentralized federated learning (DFL), adversaries can poison both model updates and the topology information devices use to choose collaborators. We present DMT…
Preserving Admission Responsibility in Multi-Tenant Large Language Model Prefix Caches
Zhiyu Wang, Rajkumar Buyya
Shared prefix caching turns Graphics Processing Unit (GPU) memory into persistent state shared across Large Language Model (LLM) tenants. A group that materializes new Key-Value (K…
PrefixPlace: Provable Prefix Key-Value Placement for Large Language Model Serving under Heterogeneous Compute and Transfer Costs
Zhiyu Wang, Rajkumar Buyya
Prefix Key-Value (KV) reuse avoids repeated prefill in Large Language Model (LLM) inference, but local misses require recomputation or replica fetches. Their relative cost varies w…
REACH: Reinforcement Learning for Adaptive Microservice Rescheduling in the Cloud-Edge Continuum
Xu Bai, Muhammed Tawfiqul Islam, Rajkumar Buyya +1
Cloud computing, despite its advantages in scalability, may not always fully satisfy the low-latency demands of emerging latency-sensitive pervasive applications. The cloud-edge co…
Service Colonies: A Novel Architectural Style for Developing Software Systems with Autonomous and Cooperative Services
Thakshila Imiya Mohottige, Artem Polyvyanyy, Rajkumar Buyya +2
This paper presents the concept of a service colony and its characteristics. A service colony is a novel architectural style for developing a software system as a group of autonomo…