4 papers
A History-Aware Visually Grounded Critic for Computer Use Agents
Jaewoo Lee, Zaid Khan, Archiki Prasad +7
Various test-time interventions for Computer Use Agents (CUAs), including critic models, have been developed to improve performance through pre-execution action evaluation in compl…
Are Video Models Zero-Shot Learners and Reasoners in Education? EduVideoBench, A Knowledge-Skills-Attitude Benchmark for Educational Video Generation
Unggi Lee, Hoyoung Ahn, Yoon Choi +16
Video generation models (VGMs) are rapidly entering classrooms, yet existing benchmarks evaluate only perceptual quality, intrinsic faithfulness, generic safety, or video as a reas…
Gistify! Codebase-Level Understanding via Runtime Execution
Hyunji Lee, Minseon Kim, Chinmay Singh +10
As coding agents are increasingly deployed in large codebases, the need to automatically design challenging, codebase-level evaluation is central. We propose Gistify, a task where…
BugPilot: Complex Bug Generation for Efficient Learning of SWE Skills
Atharv Sonwane, Isadora White, Hyunji Lee +8
High quality bugs are key to training the next generation of language model based software engineering (SWE) agents. We introduce a novel method for synthetic generation of difficu…