43 citations · 129 across the 62 of their papers we have counts for
19 papers · 1 filter
Serving Heterogeneous LoRA Adapters in Distributed LLM Inference Systems
Shashwat Jaiswal, Shrikara Arun, Anjaly Parayil +8
Low-Rank Adaptation (LoRA) has become the de facto method for parameter-efficient fine-tuning of large language models (LLMs), enabling rapid adaptation to diverse domains. In prod…
Generative Caching for Structurally Similar Prompts and Responses
Sarthak Chakraborty, Suman Nath, Xuchao Zhang +2
Large Language Models (LLMs) are increasingly being used to plan, reason, and execute tasks across diverse scenarios. In use cases like repeatable workflows and agentic settings, p…
Continuous Benchmark Generation for Evaluating Enterprise-scale LLM Agents
Divyanshu Saxena, Rishikesh Maurya, Xiaoxuan Ou +7
The rapid adoption of AI agents across domains has made systematic evaluation crucial for ensuring their usefulness and successful production deployment. Evaluation of AI agents ty…
Triage in Software Engineering: A Systematic Review of Research and Practice
Yongxin Zhao, Shenglin Zhang, Yujia Wu +5
As modern software systems continue to grow in complexity, triage has become a fundamental process in system operations and maintenance. Triage aims to efficiently prioritize, assi…
SynthAgent: Adapting Web Agents with Synthetic Supervision
Zhaoyang Wang, Yiming Liang, Xuchao Zhang +9
Web agents struggle to adapt to new websites due to the scarcity of environment specific tasks and demonstrations. Recent works have explored synthetic data generation to address t…
Can Language Models Go Beyond Coding? Assessing the Capability of Language Models to Build Real-World Systems
Chenyu Zhao, Shenglin Zhang, Zeshun Huang +8
Large language models (LLMs) have shown growing potential in software engineering, yet few benchmarks evaluate their ability to repair software during migration across instruction…
Attention Enhanced Entity Recommendation for Intelligent Monitoring in Cloud Systems
Fiza Husain, Anson Bastos, Anjaly Parayil +4
In this paper, we present DiRecGNN, an attention-enhanced entity recommendation framework for monitoring cloud services at Microsoft. We provide insights on the usefulness of this…
Anyprefer: An Agentic Framework for Preference Data Synthesis
Yiyang Zhou, Zhaoyang Wang, Tianle Wang +13
High-quality preference data is essential for aligning foundation models with human values through preference learning. However, manual annotation of such data is often time-consum…
Synergistic Weak-Strong Collaboration by Aligning Preferences
Yizhu Jiao, Xuchao Zhang, Zhaoyang Wang +7
Current Large Language Models (LLMs) excel in general reasoning yet struggle with specialized tasks requiring proprietary or domain-specific knowledge. Fine-tuning large models for…
eARCO: Efficient Automated Root Cause Analysis with Prompt Optimization
Drishti Goel, Raghav Magazine, Supriyo Ghosh +5
Root cause analysis (RCA) for incidents in large-scale cloud systems is a complex, knowledge-intensive task that often requires significant manual effort from on-call engineers (OC…
An Empirical Study of Production Incidents in Generative AI Cloud Services
Haoran Yan, Yinfang Chen, Minghua Ma +10
The ever-increasing demand for generative artificial intelligence (GenAI) has motivated cloud-based GenAI services such as Azure OpenAI Service and Amazon Bedrock. Like any large-s…
Time Warp: The Gap Between Developers' Ideal vs Actual Workweeks in an AI-Driven Era
Sukrit Kumar, Drishti Goel, Thomas Zimmermann +3
Software developers balance a variety of different tasks in a workweek, yet the allocation of time often differs from what they consider ideal. Identifying and addressing these dev…
Intent-based System Design and Operation
Vaastav Anand, Yichen Li, Alok Gautam Kumbhare +6
Cloud systems are the backbone of today's computing industry. Yet, these systems remain complicated to design, build, operate, and improve. All these tasks require significant manu…
Verifiable Format Control for Large Language Model Generations
Zhaoyang Wang, Jinqi Jiang, Huichi Zhou +4
Recent Large Language Models (LLMs) have demonstrated satisfying general instruction following ability. However, small LLMs with about 7B parameters still struggle fine-grained for…
AMPO: Active Multi-Preference Optimization for Self-play Preference Selection
Taneesh Gupta, Rahul Madhavan, Xuchao Zhang +2
Multi-preference optimization enriches language-model alignment beyond pairwise preferences by contrasting entire sets of helpful and undesired responses, thereby enabling richer t…
SageServe: Optimizing LLM Serving on Cloud Data Centers with Forecast Aware Auto-Scaling
Shashwat Jaiswal, Kunal Jain, Yogesh Simmhan +9
Global cloud service providers handle inference workloads for Large Language Models (LLMs) that span latency-sensitive (e.g., chatbots) and insensitive (e.g., report writing) tasks…
ModServe: Modality- and Stage-Aware Resource Disaggregation for Scalable Multimodal Model Serving
Haoran Qiu, Anish Biswas, Zihan Zhao +9
Large multimodal models (LMMs) demonstrate impressive capabilities in understanding images, videos, and audio beyond text. However, efficiently serving LMMs in production environme…
AIOpsLab: A Holistic Framework to Evaluate AI Agents for Enabling Autonomous Clouds
Yinfang Chen, Manish Shetty, Gagan Somashekar +6
AI for IT Operations (AIOps) aims to automate complex operational tasks, such as fault localization and root cause analysis, to reduce human workload and minimize customer impact.…
Coach: Exploiting Temporal Patterns for All-Resource Oversubscription in Cloud Platforms
Benjamin Reidys, Pantea Zardoshti, Íñigo Goiri +16
Cloud platforms remain underutilized despite multiple proposals to improve their utilization (e.g., disaggregation, harvesting, and oversubscription). Our characterization of the r…