13 papers
Carnot: Interpretable, Interactive, and Optimized Execution of Deep Research Queries
Matthew Russo, Yash Agarwal, Tianyu Li +5
Enterprises increasingly seek to query data lakes using natural language via AI-driven tools like semantic operators or deep research agents. However, the latter operates as an opa…
BEAVER: An Enterprise Benchmark for Text-to-SQL
Peter Baile Chen, Devin Yang, Weiyue Li +6
Existing text-to-SQL benchmarks have largely been constructed from public databases with well-structured schemas and simplistic question-SQL pairs. While large language models (LLM…
Agent-Aided Design for Dynamic CAD Models
Mitch Adler, Matthew Russo, Michael Cafarella
In the past year, researchers have created agentic systems that can design real-world CAD-style objects in a training-free setting, a new variety of system that we call Agent-Aided…
SAGE: Selective Attention-Guided Extraction for Token-Efficient Document Indexing
Xinzhi Wang, Peter Baile Chen, Gerardo Vitagliano +5
Large language models with long context windows can answer complex questions directly from full-length academic, technical, and policy documents, but passing entire documents is of…
OpenEstimate: Evaluating LLMs on Reasoning Under Uncertainty with Real-World Data
Alana Renda, Jillian Ross, Michael Cafarella +1
Real-world settings where language models (LMs) are deployed -- in domains spanning healthcare, finance, and other forms of knowledge work -- require models to grapple with incompl…
Abacus: A Cost-Based Optimizer for Semantic Operator Systems
Matthew Russo, Chunwei Liu, Sivaprasad Sudhir +4
LLMs enable an exciting new class of data processing applications over large collections of unstructured documents. Several new programming frameworks have enabled developers to bu…