6 papers
Measuring Agents in Production
Melissa Z. Pan, Negar Arabzadeh, Riccardo Cogo +22
LLM-based agents already operate in production across many industries, yet we lack an understanding of what technical methods make deployments successful. We present the first syst…
How to Train Your Advisor: Steering Black-Box LLMs with Advisor Models
Parth Asawa, Alan Zhu, Abigail O'Neill +3
Frontier language models are deployed as black-box services, where model weights cannot be modified and customization is limited to prompting. We introduce Advisor Models, a method…
vCache: Verified Semantic Prompt Caching
Luis Gaspar Schroeder, Aditya Desai, Alejandro Cuadron +7
Semantic caches return cached responses for semantically similar prompts to reduce LLM inference latency and cost. They embed cached prompts and store them alongside their response…
DeepScholar-Bench: A Live Benchmark and Automated Evaluation for Generative Research Synthesis
Liana Patel, Negar Arabzadeh, Harshit Gupta +4
The ability to research and synthesize knowledge is central to human expertise and progress. A new class of AI systems--designed for generative research synthesis--aims to automate…
Alto: Orchestrating Distributed Compound AI Systems with Nested Ancestry
Deepti Raghavan, Keshav Santhanam, Muhammad Shahir Rahman +7
Compound AI applications chain together subcomponents such as generative language models, document retrievers, and embedding models. Applying traditional systems optimizations such…
Semantic Operators: A Declarative Model for Rich, AI-based Data Processing
Liana Patel, Siddharth Jha, Melissa Pan +4
The semantic capabilities of large language models (LLMs) have the potential to enable rich analytics and reasoning over vast knowledge corpora. Unfortunately, existing systems eit…