Publications (23)
The science and practice of proportionality in AI risk evaluations
Carlos Mougan, Lauritz Morlock, Jair Aguirre +19
A global challenge in artificial intelligence (AI) regulation lies in achieving effective risk management without compromising innovation and technical progress. The European Union…
Inference-Time Chain-of-Thought Pruning with Latent Informativeness Signals
Sophie Li, Nicholas Huang, Nayan Saxena +4
Large language models (LLMs) improve reasoning accuracy when generating multiple candidate solutions at test time, but standard methods like Best-of-N (BoN) incur high computationa…
AgentChangeBench: A Multi-Dimensional Evaluation Framework for Goal-Shift Robustness in Conversational AI
Manik Rana, Calissa Man, Anotida Expected Msiiwa +5
Goal changes are a defining feature of real world multi-turn interactions, yet current agent benchmarks primarily evaluate static objectives or one-shot tool use. We introduce Agen…
PALADIN: Self-Correcting Language Model Agents to Cure Tool-Failure Cases
Sri Vatsa Vuddanti, Aarav Shah, Satwik Kumar Chittiprolu +4
Tool-augmented language agents frequently fail in real-world deployment due to tool malfunctions--timeouts, API exceptions, or inconsistent outputs--triggering cascading reasoning…
Visualizing and Benchmarking LLM Factual Hallucination Tendencies via Internal State Analysis and Clustering
Nathan Mao, Varun Kaushik, Shreya Shivkumar +3
Large Language Models (LLMs) often hallucinate, generating nonsensical or false information that can be especially harmful in sensitive fields such as medicine or law. To study thi…
Open Weight AI Models Require Proportional Evaluation Approaches
Patricia Paskov, Christopher Rodriguez, Sunishchal Dev +1
Open-weight AI models (OWMs), or models released with publicly-available weights, are distributing rapidly and approaching the performance levels of leading closed-weight AI models…