#benchmark
86 papers match
EEG-EditBench: Probing Visual Information in EEG-Image Retrieval Models with Controlled Image Edits
Kaifan Zhang, Lihuo He, Yuqi Ji +4
The paper presents EEG-EditBench, a benchmark that uses controlled image edits to evaluate how EEG-to-image retrieval models capture fine-grained visual information.
OSReward: Instituting Standardized Evaluation for Cross-Platform Computer-Use Reward Models
Qiushi Sun, Kanzhi Cheng, Yian Wang +20
The paper introduces OSReward, a benchmark for evaluating vision-language model judges that assess computer-using agent trajectories, and presents open reward models (OS‑Shepherd)…
Can LVLMs Uncover the Truth Behind Visual Illusions? An Analysis of Perceptual and Reasoning Capabilities
Liangjie Zhao, Jiaqing Lyu, Kexin Tang +5
The paper introduces IllusionReasoning, a benchmark that uses visual illusion images to jointly assess perception and reasoning abilities of large vision‑language models, revealing…
BlueprintRepair: Typed Local Edits for Failed Lean Proof Blueprints
Ruslan Khrulev
The paper introduces BlueprintRepair, an interface that lets large language models make typed, local edits to Lean proof blueprints, and evaluates it on a benchmark of controlled p…
From Single- to Cross-Document: Benchmarking Multi-Granularity Event Analysis of Large Language Models
Tao Wen, Shuai Shao, Pei Ke +7
The paper introduces MiGUE-Bench, a benchmark that evaluates large language models on multi‑granularity event analysis tasks ranging from single‑document event detection to cross‑d…
IFHierBench: Hierarchical Instruction Following for Large Language Models
Yuetian Mao, Chunyang Chen
The paper introduces IFHierBench, a benchmark for evaluating how well large language models follow hierarchical, nested constraints in instruction prompts, and shows current models…