6 papers
VAKRA: Evaluating Multi-Hop Reasoning Across APIs and Retrieval Under Tool-Use Policies
Ankita Rajaram Naik, Anupama Murthi, Benjamin Elder +6
Agents deployed in enterprise settings must reason across structured APIs and document collections, yet existing benchmarks evaluate these capabilities in isolation. We introduce V…
Adaptive Turn-Taking for Real-time Multi-Party Voice Agents
Soumyajit Mitra, Prabhat Pandey, Abhinav Jain +2
Turn-taking in multi-party spoken conversations remains a fundamental challenge for voice-based agents, particularly under dynamic floor competition and varying user expectations.…
MHA-RAG: Improving Efficiency, Accuracy, and Consistency by Encoding Exemplars as Soft Prompts
Abhinav Jain, Xinyu Yao, Thomas Reps +1
Adapting Foundation Models to new domains with limited training data is challenging and computationally expensive. While prior work has demonstrated the effectiveness of using doma…
DOPPLER: Dual-Policy Learning for Device Assignment in Asynchronous Dataflow Graphs
Xinyu Yao, Daniel Bourgeois, Abhinav Jain +5
We study the problem of assigning operations in a dataflow graph to devices to minimize execution time in a work-conserving system, with emphasis on complex machine learning worklo…
End-to-End Text-to-SQL with Dataset Selection: Leveraging LLMs for Adaptive Query Generation
Anurag Tripathi, Vaibhav Patle, Abhinav Jain +4
Text-to-SQL bridges the gap between natural language and structured database language, thus allowing non-technical users to easily query databases. Traditional approaches model tex…
SIMCOPILOT: Evaluating Large Language Models for Copilot-Style Code Generation
Mingchao Jiang, Abhinav Jain, Sophia Zorek +1
We introduce SIMCOPILOT, a benchmark that simulates the role of large language models (LLMs) as interactive, "copilot"-style coding assistants. Targeting both completion (finishing…