6 papers
TRACE-ROUTER: Task-Consistent and Adaptive Online Routing for Agentic AI
Ritik Raj, Souvik Kundu, Sarbartha Banerjee +3
Routing to select large language models (LLMs) with different cost-quality trade-offs has become a fundamental deployment feature of enterprise AI. Existing routers, primarily make…
How Far Can Disaggregation Go? A Design-Space Exploration of Attention-FFN Disaggregation for Efficient MoE LLM Serving
Hanjiang Wu, Abhimanyu Rajeshkumar Bambhaniya, Sarbartha Banerjee +9
Modern large language model (LLM) inference has progressively disaggregated to keep pace with growing model sizes and tight TTFT and TPOT service-level objectives: from chunked-pre…
Cascade: Composing Software-Hardware Attack Gadgets for Adversarial Threat Amplification in Compound AI Systems
Sarbartha Banerjee, Prateek Sahu, Anjo Vahldiek-Oberwagner +2
Rapid progress in generative AI has given rise to Compound AI systems - pipelines comprised of multiple large language models (LLM), software tools and database systems. Compound A…
SCALE-Sim v3: A modular cycle-accurate systolic accelerator simulator for end-to-end system analysis
Ritik Raj, Sarbartha Banerjee, Nikhil Chandra +4
The rapid advancements in AI, scientific computing, and high-performance computing (HPC) have driven the need for versatile and efficient hardware accelerators. Existing tools like…
SoK: A Systems Perspective on Compound AI Threats and Countermeasures
Sarbartha Banerjee, Prateek Sahu, Mulong Luo +3
Large language models (LLMs) used across enterprises often use proprietary models and operate on sensitive inputs and data. The wide range of attack vectors identified in prior res…
ConfusedPilot: Confused Deputy Risks in RAG-based LLMs
Ayush RoyChowdhury, Mulong Luo, Prateek Sahu +2
Retrieval augmented generation (RAG) is a process where a large language model (LLM) retrieves useful information from a database and then generates the responses. It is becoming p…