Showing cs.AIShow all
3 papers · 1 filter
cs.AI2025
Autonomous Assessment of Generalizability of AI Agent Capabilities
Daniel Bramblett, Rushang Karia, Adrian Ciotinga +3
Safe deployment of black-box AI (BBAI) systems such as foundation model agents requires methods for evaluating their capabilities in novel settings. We define an agent's capability…
cs.AI2024
Autonomous Evaluation of LLMs for Truth Maintenance and Reasoning Tasks
Rushang Karia, Daniel Bramblett, Daksh Dobhal +1
This paper presents AutoEval, a novel benchmark for scaling Large Language Model (LLM) assessment in formal tasks with clear notions of correctness, such as truth maintenance in tr…
cs.AI2024
Belief-State Query Policies for User-Aligned POMDPs
Daniel Bramblett, Siddharth Srivastava
Planning in real-world settings often entails addressing partial observability while aligning with users' requirements. We present a novel framework for expressing users' constrain…