47 citations · 60 across the 14 of their papers we have counts for
14 papers
Indi-RomCoM: Code-Mixed Benchmark for Evaluating LLMs on Romanized Indic-English Instructions
Avisha Das, Mihir Parmar, Mohana Ramnath +1
Romanized Code Mixing (RCM), where bilingual speakers fluidly blend local languages with English in Roman script, has emerged as the dominant form of communication across multiling…
Autonomous Assessment of Generalizability of AI Agent Capabilities
Daniel Bramblett, Rushang Karia, Adrian Ciotinga +3
Safe deployment of black-box AI (BBAI) systems such as foundation model agents requires methods for evaluating their capabilities in novel settings. We define an agent's capability…
AI Planning: A Primer and Survey (Preliminary Report)
Dillon Z. Chen, Pulkit Verma, Siddharth Srivastava +2
Automated decision-making is a fundamental topic that spans multiple sub-disciplines in AI: reinforcement learning (RL), AI planning (AP), foundation models, and operations researc…
Using Explainable AI and Hierarchical Planning for Outreach with Robots
Rushang Karia, Jayesh Nagpal, Daksh Dobhal +4
Understanding how robots plan and execute tasks is crucial in today's world, where they are becoming more prevalent in our daily lives. However, teaching non-experts, such as K-12…
utoval: Autonomous Assessment of LLMs in Formal Synthesis and Interpretation Tasks
Rushang Karia, Daniel Bramblett, Daksh Dobhal +2
This paper presents utoval, a new approach for scaling LLM assessment in translating formal syntax -- such as first-order logic, regular expressions, etc -- to na…
From Real World to Logic and Back: Learning Generalizable Relational Concepts For Long Horizon Robot Planning
Naman Shah, Jayesh Nagpal, Siddharth Srivastava
Robots still lag behind humans in their ability to generalize from limited experience, particularly when transferring learned behaviors to long-horizon tasks in unseen environments…
Epistemic Exploration for Generalizable Planning and Learning in Non-Stationary Settings
Rushang Karia, Pulkit Verma, Alberto Speranzon +1
This paper introduces a new approach for continual planning and model learning in relational, non-stationary stochastic environments. Such capabilities are essential for the deploy…
Autonomous Capability Assessment of Sequential Decision-Making Systems in Stochastic Settings (Extended Version)
Pulkit Verma, Rushang Karia, Siddharth Srivastava
It is essential for users to understand what their AI systems can and can't do in order to use them safely. However, the problem of enabling users to assess AI systems with sequent…
Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks
Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi +37
How well can NLP models generalize to a variety of unseen tasks when provided with task instructions? To address this question, we first introduce Super-NaturalInstructions, a benc…
Differential Assessment of Black-Box AI Agents
Rashmeet Kaur Nayyar, Pulkit Verma, Siddharth Srivastava
Much of the research on learning symbolic models of AI agents focuses on agents with stationary models. This assumption fails to hold in settings where the agent's capabilities may…
JEDAI: A System for Skill-Aligned Explainable Robot Planning
Naman Shah, Pulkit Verma, Trevor Angle +1
This paper presents JEDAI, an AI system designed for outreach and educational efforts aimed at non-AI experts. JEDAI features a novel synthesis of research ideas from integrated ta…
Learning Causal Models of Autonomous Agents using Interventions
Pulkit Verma, Siddharth Srivastava
One of the several obstacles in the widespread use of AI systems is the lack of requirements of interpretability that can enable a layperson to ensure the safe and reliable behavio…
Discovering User-Interpretable Capabilities of Black-Box Planning Agents
Pulkit Verma, Shashank Rao Marpally, Siddharth Srivastava
Several approaches have been developed for answering users' specific questions about AI behavior and for assessing their core functionality in terms of primitive executable actions…
Asking the Right Questions: Learning Interpretable Action Models Through Query Answering
Pulkit Verma, Shashank Rao Marpally, Siddharth Srivastava
This paper develops a new approach for estimating an interpretable, relational model of a black-box autonomous agent that can plan and act. Our main contributions are a new paradig…