11 citations · 19 across the 11 of their papers we have counts for
11 papers
Adversarial Course-of-Action Generation: Game-Theoretic Multi-Agent Algorithms for COA matching & COA generation
Natan Vidra, Alina Kapanova, Arun Kanhai +1
Course-of-action (COA) generation is a distributed planning problem: a system must propose structured candidate actions, evaluate them against an adversarial response, and surface…
Cross Lingual Transfer in Tulu Legal Comprehension: Script-Dependent Improvement and RAG-Induced Knowledge Conflict
Sindhu Shetty, Spurthi Setty, Natan Vidra
Low-resource languages without an adequate training corpus often use a related, higher-resource language as a scaffold for comprehension. Still, there is a need to develop rigorous…
Beyond Pass@k: Measuring Reliability and Security of Agentic Code Generation
Jiajun Jiang, Sharon Zheng, Natan Vidra +1
AI coding agent benchmarks rank agents with the Chen et al. (2021) pass@k estimator, but current implementations misapply it: they set n to the number of unit tests in a single sub…
DevIntent: How Much Does LLM-Generated Code Violate Developer Intent?
Susana Haing, Natan Vidra, Spurthi Setty
Code generated by LLMs can violate a developer's implicit intentions when given an ambiguous prompt, yet standard benchmarks measure only whether code passes its stated test. We in…
OrchestraBench: Evaluating Multi-Agent Orchestration Failure Modes, Recovery, and Decomposition Quality
Yidian Chen, Yingzi Gu, Natan Vidra +2
Multi-agent orchestration frameworks are moving from demos to production, yet benchmarks typically report task accuracy without diagnosing why a pipeline failed, where a cascade be…
Posture and Sustainment Optimization Under Adversarial Uncertainty
Amelie Norris, Alyssa Lee, Natan Vidra +1
Pre-commitment posture, the assignment of military assets to theater locations before conflict scenarios resolve, is a critical and formally unsolved problem in joint operational p…
AdvPlan-Bench: Adversarial Evaluation of Structured Plan-Generation Agents
Alina Kapanova, Arun Kanhai, Natan Vidra +1
Structured plan-generation agents are often evaluated as if a plan has quality in isolation, yet many realistic planning tasks require asking how a candidate behaves when another a…
MetaRoute-Bench: Evaluating Meta-Decision Policies for Agentic Workflow Routing
Natan Vidra, Alina Kapanova, Arun Kanhai +1
Agentic systems must repeatedly decide whether to answer directly, decompose a task, invoke a tool, execute code, delegate to a specialist, verify an intermediate result, or recove…
Learning Compositional Meta-Routing for Agentic Workflows: An Executable Benchmark
Natan Vidra, Alina Kapanova, Arun Kanhai +1
Agentic systems must decide not only what answer to produce, but which reasoning and execution operations should precede it. A controller may answer directly, decompose a request,…
Improving Retrieval for RAG based Question Answering Models on Financial Documents
Spurthi Setty, Harsh Thakkar, Alyssa Lee +2
The effectiveness of Large Language Models (LLMs) in generating accurate responses relies heavily on the quality of input provided, particularly when employing Retrieval Augmented…
Enhancing Large Language Model Performance To Answer Questions and Extract Information More Accurately
Liang Zhang, Katherine Jijo, Spurthi Setty +4
Large Language Models (LLMs) generate responses to questions; however, their effectiveness is often hindered by sub-optimal quality of answers and occasional failures to provide ac…