6 citations · 17 across the 14 of their papers we have counts for
16 papers
Koi: Evidence-Guided, Self-Calibrating Planning for Multi-Cloud, Heterogeneous LLM Inference Fleets
Mankeerat Sidhu, Gangmuk Lim, Hetarth Chopra +10
Deploying a large language model (LLM) requires choosing the right accelerator, parallelization strategy, engine configuration, and replica count amongst ~140 other variables acros…
DuoDiscover: Scaling Autonomous Discovery Beyond the Evaluation Bottleneck
Shubham Agarwal, Akshat Gupta, Mike Heddes +21
Evaluation has been central to advances in AI, and it is equally critical to autonomous discovery. Yet scaling autonomous discovery to real-world engineering challenges faces a fun…
Fantastic Adaptive Taxonomies and How to Use Them
Mert Cemri, Andrei Cojocaru, Melissa Pan +9
An agent system's execution traces record how it fails, and procedures that improve such a system without changing model weights (trajectory selection, prompt and workflow optimiza…
BenchEvolver: Frontier Task Synthesis via Solution-Centric Evolution
Yangzhen Wu, Aaron J. Li, Wenjie Ma +10
The rapid progress of frontier large language models has led to widespread benchmark saturation, limiting the ability of existing datasets to differentiate model capabilities or pr…
The Time is Here for Just-in-Time Systems: Challenges and Opportunities
Shu Liu, Alexander Krentsel, Shubham Agarwal +8
Core systems like key-value stores have historically taken years to build, and are designed to be general so as to amortize cost across deployments, paying a significant performanc…
Inductive Deductive Synthesis: Enabling AI to Generate Formally Verified Systems
Shubham Agarwal, Alexander Krentsel, Shu Liu +10
AI agents increasingly excel at generating, testing, and refining code. However, they fall short on tasks requiring formal guarantees of full coverage that testing alone cannot pro…
AdaEvolve: Adaptive LLM Driven Zeroth-Order Optimization
Mert Cemri, Shubham Agrawal, Akshat Gupta +9
The paradigm of automated program generation is shifting from one-shot generation to inference-time search, where Large Language Models (LLMs) function as semantic mutation operato…
Measuring Agents in Production
Melissa Z. Pan, Negar Arabzadeh, Riccardo Cogo +22
LLM-based agents already operate in production across many industries, yet we lack an understanding of what technical methods make deployments successful. We present the first syst…
SkyRL-Agent: Efficient RL Training for Multi-turn LLM Agent
Shiyi Cao, Dacheng Li, Fangzhou Zhao +12
We introduce SkyRL-Agent, a framework for efficient, multi-turn, long-horizon agent training and evaluation. It provides efficient asynchronous dispatching, lightweight tool integr…
LEANN: A Low-Storage Vector Index
Yichuan Wang, Zhifei Li, Shu Liu +10
Embedding-based vector search underpins many important applications, such as recommendation and retrieval-augmented generation (RAG). It relies on vector indices to enable efficien…
Jenga: Effective Memory Management for Serving LLM with Heterogeneity
Chen Zhang, Kuntai Du, Shu Liu +10
Large language models (LLMs) are widely used but expensive to run, especially as inference workloads grow. To lower costs, maximizing the request batch size by managing GPU memory…
Locality-aware Fair Scheduling in LLM Serving
Shiyi Cao, Yichuan Wang, Ziming Mao +10
Large language model (LLM) inference workload dominates a wide variety of modern AI applications, ranging from multi-turn conversation to document analysis. Balancing fairness and…
MoE-Lightning: High-Throughput MoE Inference on Memory-constrained GPUs
Shiyi Cao, Shu Liu, Tyler Griggs +6
Efficient deployment of large language models, particularly Mixture of Experts (MoE), on resource-constrained platforms presents significant challenges, especially in terms of comp…
Pie: Pooling CPU Memory for LLM Inference
Yi Xu, Ziming Mao, Xiangxi Mo +2
The rapid growth of LLMs has revolutionized natural language processing and AI analysis, but their increasing size and memory demands present significant challenges. A common solut…
Text2SQL is Not Enough: Unifying AI and Databases with TAG
Asim Biswal, Liana Patel, Siddarth Jha +5
AI systems that serve natural language questions over databases promise to unlock tremendous value. Such systems would allow users to leverage the powerful reasoning and knowledge…
Optimizing LLM Queries in Relational Data Analytics Workloads
Shu Liu, Asim Biswal, Amog Kamsetty +8
Batch data analytics is a growing application for Large Language Models (LLMs). LLMs enable users to perform a wide range of natural language tasks, such as classification, entity…