2 citations · 7 across the 15 of their papers we have counts for
16 papers
ORACLE: Agentic AI Orchestrator Routing Via Adaptive Verifier Calibration Feedback
Ritik Raj, Souvik Kundu, Sarbartha Banerjee +3
Modern enterprise agent deployments consist of a heterogeneous pool of large language models (LLMs) having diverse capabilities and cost. Existing model routing strategies optimize…
Serving Agentic Workflows with a Physical-Plan Compiler and Adaptive Runtime
Jiayi Qian, Yichong Zhang, Zishen Wan +4
Efficient serving of agentic workflows requires selecting each LLM node's model, verification policy, and backend to balance output quality, latency, and throughput. These assignme…
Polestar: Drift-Aware Cache Calibration and Token Commitment for Efficient Inference of Diffusion LLMs
Mingyu Lee, Akshat Ramachandran, Souvik Kundu +1
The inference efficiency of diffusion large language models (dLLMs) is constrained by two challenges: bidirectional attention precludes efficient KV-cache reuse, while increasing d…
How Far Can Disaggregation Go? A Design-Space Exploration of Attention-FFN Disaggregation for Efficient MoE LLM Serving
Hanjiang Wu, Abhimanyu Rajeshkumar Bambhaniya, Sarbartha Banerjee +9
Modern large language model (LLM) inference has progressively disaggregated to keep pace with growing model sizes and tight TTFT and TPOT service-level objectives: from chunked-pre…
Towards Understanding, Analyzing, and Optimizing Agentic AI Execution: A CPU-Centric Perspective
Ritik Raj, Souvik Kundu, Ishita Vohra +2
Agentic AI serving converts monolithic LLM-based inference to autonomous problem-solvers that can plan, call tools, perform reasoning, and adapt on the fly. Due to diverse task exe…
On Evaluating Performance of LLM Inference Serving Systems
Amey Agrawal, Nitin Kedia, Anmol Agarwal +5
The rapid evolution of Large Language Model (LLM) inference systems has yielded significant efficiency improvements. However, our systematic analysis reveals that current evaluatio…
MIST: A Co-Design Framework for Heterogeneous, Multi-Stage LLM Inference
Abhimanyu Rajeshkumar Bambhaniya, Hanjiang Wu, Suvinay Subramanian +8
Modern LLM serving now spans multi-stage pipelines including RAG retrieval and KV cache reuse, each with distinct compute, memory, and latency demands. Inference engines expose a l…
MoDM: Efficient Serving for Image Generation via Mixture-of-Diffusion Models
Yuchen Xia, Divyam Sharma, Yichao Yuan +2
Diffusion-based text-to-image generation models trade latency for quality: small models are fast but generate lower-quality images, while large models produce better images but are…
OuroMamba: A Data-Free Quantization Framework for Vision Mamba
Akshat Ramachandran, Mingyu Lee, Huan Xu +2
We present OuroMamba, the first data-free post-training quantization (DFQ) method for vision Mamba-based models (VMMs). We identify two key challenges in enabling DFQ for VMMs, (1)…
MicroScopiQ: Accelerating Foundational Models through Outlier-Aware Microscaling Quantization
Akshat Ramachandran, Souvik Kundu, Tushar Krishna
Quantization of foundational models (FMs) is significantly more challenging than traditional DNNs due to the emergence of large magnitude values called outliers. Existing outlier-a…
Etalon: Holistic Performance Evaluation Framework for LLM Inference Systems
Amey Agrawal, Anmol Agarwal, Nitin Kedia +5
Serving large language models (LLMs) in production can incur substantial costs, which has prompted recent advances in inference system optimizations. Today, these systems are evalu…
Understanding the Performance and Estimating the Cost of LLM Fine-Tuning
Yuchen Xia, Jiho Kim, Yuhan Chen +4
Due to the cost-prohibitive nature of training Large Language Models (LLMs), fine-tuning has emerged as an attractive alternative for specializing LLMs for specific tasks using lim…
CLAMP-ViT: Contrastive Data-Free Learning for Adaptive Post-Training Quantization of ViTs
Akshat Ramachandran, Souvik Kundu, Tushar Krishna
We present CLAMP-ViT, a data-free post-training quantization method for vision transformers (ViTs). We identify the limitations of recent techniques, notably their inability to lev…
ShiftAddLLM: Accelerating Pretrained LLMs via Post-Training Multiplication-Less Reparameterization
Haoran You, Yipin Guo, Yichao Fu +6
Large language models (LLMs) have shown impressive performance on language tasks but face challenges when deployed on resource-constrained devices due to their extensive parameters…
Demystifying AI Platform Design for Distributed Inference of Next-Generation LLM models
Abhimanyu Bambhaniya, Ritik Raj, Geonhwa Jeong +6
Large language models (LLMs) have shown remarkable performance across a wide range of applications, often outperforming human experts. However, deploying these gigantic models effi…
CiMNet: Towards Joint Optimization for DNN Architecture and Configuration for Compute-In-Memory Hardware
Souvik Kundu, Anthony Sarah, Vinay Joshi +2
With the recent growth in demand for large-scale deep neural networks, compute in-memory (CiM) has come up as a prominent solution to alleviate bandwidth and on-chip interconnect b…