43 citations · 129 across the 62 of their papers we have counts for
14 papers · 1 filter
A Resource-centric Analysis and Optimization of NoSQL Workloads using Distressed Resource Volume Metric
Gunika Verma, Aashutosh A, Pooja Srinivas +9
Large-scale managed cloud databases leverage sophisticated load Packing and Migration (PAM) algorithms, which provide the efficiencies necessary for running these services at scale…
A Holistic Framework for Automated Configuration Recommendation for Cloud Service Monitoring
Anson Bastos, Shreeya Venneti, Anjaly Parayil +3
Reliability of large-scale cloud services is critical for user satisfaction and business continuity. Despite significant investments in reliability engineering, production incident…
Sutradhara: An Intelligent Orchestrator-Engine Co-design for Tool-based Agentic Inference
Anish Biswas, Kanishk Goel, Srivarshinee S +5
Agentic applications are LLMs that iteratively invoke external tools to accomplish complex tasks. Such tool-based agents are rapidly becoming the dominant paradigm for deploying la…
Serving Heterogeneous LoRA Adapters in Distributed LLM Inference Systems
Shashwat Jaiswal, Shrikara Arun, Anjaly Parayil +8
Low-Rank Adaptation (LoRA) has become the de facto method for parameter-efficient fine-tuning of large language models (LLMs), enabling rapid adaptation to diverse domains. In prod…
An Empirical Study of Production Incidents in Generative AI Cloud Services
Haoran Yan, Yinfang Chen, Minghua Ma +10
The ever-increasing demand for generative artificial intelligence (GenAI) has motivated cloud-based GenAI services such as Azure OpenAI Service and Amazon Bedrock. Like any large-s…
Intent-based System Design and Operation
Vaastav Anand, Yichen Li, Alok Gautam Kumbhare +6
Cloud systems are the backbone of today's computing industry. Yet, these systems remain complicated to design, build, operate, and improve. All these tasks require significant manu…
SageServe: Optimizing LLM Serving on Cloud Data Centers with Forecast Aware Auto-Scaling
Shashwat Jaiswal, Kunal Jain, Yogesh Simmhan +9
Global cloud service providers handle inference workloads for Large Language Models (LLMs) that span latency-sensitive (e.g., chatbots) and insensitive (e.g., report writing) tasks…
ModServe: Modality- and Stage-Aware Resource Disaggregation for Scalable Multimodal Model Serving
Haoran Qiu, Anish Biswas, Zihan Zhao +9
Large multimodal models (LMMs) demonstrate impressive capabilities in understanding images, videos, and audio beyond text. However, efficiently serving LMMs in production environme…
Intelligent Router for LLM Workloads: Improving Performance Through Workload-Aware Load Balancing
Kunal Jain, Anjaly Parayil, Ankur Mallick +10
Large Language Model (LLM) workloads have distinct prefill and decode phases with different compute and memory requirements which should ideally be accounted for when scheduling in…
Towards Cloud Efficiency with Large-scale Workload Characterization
Anjaly Parayil, Jue Zhang, Xiaoting Qin +4
Cloud providers introduce features (e.g., Spot VMs, Harvest VMs, and Burstable VMs) and optimizations (e.g., oversubscription, auto-scaling, power harvesting, and overclocking) to…
Workload Intelligence: Punching Holes Through the Cloud Abstraction
Lexiang Huang, Anjaly Parayil, Jue Zhang +13
Today, cloud workloads are essentially opaque to the cloud platform. Typically, the only information the platform receives is the virtual machine (VM) type and possibly a decoratio…
Dependency Aware Incident Linking in Large Cloud Systems
Supriyo Ghosh, Karish Grover, Jimmy Wong +4
Despite significant reliability efforts, large-scale cloud services inevitably experience production incidents that can significantly impact service availability and customer's sat…
DeCaf: Diagnosing and Triaging Performance Issues in Large-Scale Cloud Services
Chetan Bansal, Sundararajan Renganathan, Ashima Asudani +2
Large scale cloud services use Key Performance Indicators (KPIs) for tracking and monitoring performance. They usually have Service Level Objectives (SLOs) baked into the customer…
Static Analysis Using the Cloud
Rahul Kumar, Chetan Bansal, Jakob Lichtenberg
In this paper we describe our experience of using Microsoft Azure cloud computing platform for static analysis. We start by extending Static Driver Verifier to operate in the Micro…