25 citations · 29 across the 15 of their papers we have counts for
4 papers · 1 filter
SpeedrunBench: Challenging LLM Agents with Video Game Speedrunning
Yoshinari Fujinuma, Keisuke Kamahori, Ryuto Koike +8
Frontier LLM agents have been shown to be capable of solving increasingly complex tasks for which humans have measurable solutions. This begs the pertinent question of whether LLM…
MEMTRACK: Evaluating Long-Term Memory and State Tracking in Multi-Platform Dynamic Agent Environments
Darshan Deshpande, Varun Gangal, Hersh Mehta +3
Recent works on context and memory benchmarking have primarily focused on conversational instances but the need for evaluating memory in dynamic enterprise environments is crucial…
TRAIL: Trace Reasoning and Agentic Issue Localization
Darshan Deshpande, Varun Gangal, Hersh Mehta +3
The increasing adoption of agentic workflows across diverse domains brings a critical need to scalably and systematically evaluate the complex traces these systems generate. Curren…
Enhancing Hallucination Detection through Perturbation-Based Synthetic Data Generation in System Responses
Dongxu Zhang, Varun Gangal, Barrett Martin Lattimer +1
Detecting hallucinations in large language model (LLM) outputs is pivotal, yet traditional fine-tuning for this classification task is impeded by the expensive and quickly outdated…