3 papers
cs.AI2026
Implicit Intelligence -- Evaluating Agents on What Users Don't Say
Ved Sirdeshmukh, Marc Wetter
Real-world requests to AI agents are fundamentally underspecified. Natural human communication relies on shared context and unstated constraints that speakers expect listeners to i…
cs.CY2025
FORTRESS: Frontier Risk Evaluation for National Security and Public Safety
Christina Q. Knight, Kaustubh Deshpande, Ved Sirdeshmukh +4
The rapid advancement of large language models (LLMs) introduces dual-use capabilities that could both threaten and bolster national security and public safety (NSPS). Models imple…
cs.CL2025
MultiChallenge: A Realistic Multi-Turn Conversation Evaluation Benchmark Challenging to Frontier LLMs
Ved Sirdeshmukh, Kaustubh Deshpande, Johannes Mols +7
We present MultiChallenge, a pioneering benchmark evaluating large language models (LLMs) on conducting multi-turn conversations with human users, a crucial yet underexamined capab…