Showing cs.AIShow all
3 papers · 1 filter
cs.AI2025
Monte Carlo Query Search: Active Capability Assessment of AI Agents
Daniel Bramblett, Rushang Karia, Adrian Ciotinga +3
Black-box AI (BBAI) systems, including foundation-model agents, are increasingly used for sequential decision making. Safe deployment requires methods for characterizing what such…
cs.AI2024
Autonomous Evaluation of LLMs for Truth Maintenance and Reasoning Tasks
Rushang Karia, Daniel Bramblett, Daksh Dobhal +1
This paper presents AutoEval, a novel benchmark for scaling Large Language Model (LLM) assessment in formal tasks with clear notions of correctness, such as truth maintenance in tr…
cs.AI2024
Belief-State Query Policies for User-Aligned POMDPs
Daniel Bramblett, Siddharth Srivastava
Planning in real-world settings often entails addressing partial observability while aligning with users' requirements. We present a novel framework for expressing users' constrain…