5 papers
Evaluating from Benign to Dynamic Adversarial: A Squid Game for Large Language Models
Zijian Chen, Wenjun Zhang, Guangtao Zhai
The potential data contamination issue in contemporary large language models (LLMs) benchmarks presents a fundamental challenge to establishing trustworthy evaluation frameworks. M…
MACEval: A Multi-Agent Continual Evaluation Network for Large Models
Zijian Chen, Yuze Sun, Yuan Tian +2
Hundreds of benchmarks dedicated to evaluating large models have been presented over the past few years. However, most of them remain closed-ended and are prone to overfitting due…
Just Noticeable Difference for Large Multimodal Models
Zijian Chen, Yuan Tian, Yuze Sun +5
Just noticeable difference (JND), the minimum change that the human visual system (HVS) can perceive, has been studied for decades. Although recent work has extended this line of r…
OBI-Bench: Can LMMs Aid in Study of Ancient Script on Oracle Bones?
Zijian Chen, Tingzhu Chen, Wenjun Zhang +1
We introduce OBI-Bench, a holistic benchmark crafted to systematically evaluate large multi-modal models (LMMs) on whole-process oracle bone inscriptions (OBI) processing tasks dem…
GAIA: Rethinking Action Quality Assessment for AI-Generated Videos
Zijian Chen, Wei Sun, Yuan Tian +7
Assessing action quality is both imperative and challenging due to its significant impact on the quality of AI-generated videos, further complicated by the inherently ambiguous nat…