1 citations · 1 across the 21 of their papers we have counts for
26 papers
From Pixel to Coding: Evaluating the Figure Reproduction Capabilities of MLLMs
Zijian Chen, Zhengyu Chen, Bohan Liang +7
Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities in both visual understanding and code generation. However, existing benchmarks typically evaluate…
Zero2Repo: Can Coding Agents Build Repositories from Scratch?
Pei Yang, Tianyu Shi, Yuhang Yao +23
Coding agents are increasingly asked to build software rather than patch it, yet benchmarks for from-scratch repository construction are mostly limited to a single language and dep…
Learning from Others, Acting for You: Cross-User Memory Sharing for LLM Agents
Jinming Hu, Haodong Zhao, Qi Jia +4
Large language model (LLM) agents serving different users often solve related tasks, yet separate user histories can leave reusable experience inaccessible to other agents. Pooling…
PRISM-Bench: An Audio-Centric Diagnostic Benchmark for Text-to-Audio-Video Generation
Yuchen Sun, Qian Yang, Jun Wang +4
Text-to-audio-video (T2AV) generation has advanced rapidly, but its evaluation still underestimates the audio modality. Existing benchmarks either treat audio as an auxiliary compo…
SafeAtlas-VL: Beyond Binary Multimodal Safety with Large-Scale Data and Guard Models
Zongrui Wang, Xiangyang Zhu, Sicheng Wang +13
Multimodal safety moderation requires distinguishing risks arising from visual content, user intent, and assistant behavior. Existing safeguards, however, are typically trained for…
SciMIF: Understanding Multimodal Instruction Following in Scientific Domains
Ye Shen, Yuting Zheng, Dun Pei +4
Understanding instruction-following capabilities in scientific domains is essential for effectively leveraging Multimodal Large Language Models (MLLMs) to advance the development o…
Towards High-Level Semantic Intelligence
Xiujie Song, Gefei Yang, Yining You +6
Recent advances in AI have substantially expanded its cognitive and reasoning capabilities. From the perspective of semantic complexity, the development of AI reveals a clear traje…
LatentRevise: Learning from Zero-Hit Reasoning
Yiqiu Guo, Xueting Han, Qi Jia +2
Reinforcement learning with verifiable rewards (RLVR) is bottlenecked by hard prompts on which correct trajectories have low probability, so sampling misses them within a practical…
Emergent Misalignment Can Be Induced by Sycophancy and Reversed via Alignment Gating
Sicheng Wang, Xiangyang Zhu, Han Wang +6
Prior work has shown that fine-tuning large language models on malicious or incorrect outputs in narrow domains can induce broad misalignment and harmful behavior, a phenomenon kno…
SafeSci: Safety Evaluation of Large Language Models in Science Domains and Beyond
Xiangyang Zhu, Yuan Tian, Qi Jia +14
The success of large language models (LLMs) in scientific domains has heightened safety concerns, prompting numerous benchmarks to evaluate their scientific safety. Existing benchm…
UniDial-EvalKit: A Unified Toolkit for Evaluating Multi-Faceted Conversational Abilities
Qi Jia, Haodong Zhao, Dun Pei +7
Benchmarking large language models (LLMs) and agents in multi-turn interactive scenarios is essential for understanding their practical capabilities. However, existing evaluation p…
VideoAesBench: Benchmarking the Video Aesthetics Perception Capabilities of Large Multimodal Models
Yunhao Li, Sijing Wu, Zhilin Gao +5
Large multimodal models (LMMs) have demonstrated outstanding capabilities in various visual perception tasks, which has in turn made the evaluation of LMMs significant. However, th…
Q-Bench-Portrait: Benchmarking Multimodal Large Language Models on Portrait Image Quality Perception
Sijing Wu, Yunhao Li, Zicheng Zhang +5
Recent advances in multimodal large language models (MLLMs) have demonstrated impressive performance on existing low-level vision benchmarks, which primarily focus on generic image…
KidVis: Do Multimodal Large Language Models Possess the Visual Perceptual Capabilities of a 6-Year-Old?
Xianfeng Wang, Kaiwei Zhang, Qi Jia +3
While Multimodal Large Language Models (MLLMs) have demonstrated impressive proficiency in high-level reasoning tasks, such as complex diagrammatic interpretation, it remains an op…
EvolMem: A Cognitive-Driven Benchmark for Multi-Session Dialogue Memory
Ye Shen, Dun Pei, Yiqiu Guo +6
Despite recent advances in understanding and leveraging long-range conversational memory, existing benchmarks still lack systematic evaluation of large language models(LLMs) across…
PriceSeer: Evaluating Large Language Models in Real-Time Stock Prediction
Bohan Liang, Zijian Chen, Qi Jia +3
Stock prediction, a subject closely related to people's investment activities in fully dynamic and live environments, has been widely studied. Current large language models (LLMs)…
LiveProteinBench: A Contamination-Free Benchmark for Assessing Models' Specialized Capabilities in Protein Science
Dingyi Rong, Zijian Chen, Qi Jia +4
In contrast to their remarkable performance on general knowledge QA, the true abilities of Large Language Models (LLMs) in tasks demanding deep, specialized reasoning, such as in p…
One Battle After Another: Probing LLMs' Limits on Multi-Turn Instruction Following with a Benchmark Evolving Framework
Qi Jia, Ye Shen, Xiujie Song +5
Evaluating LLMs' instruction-following ability in multi-topic dialogues is essential yet challenging. Existing benchmarks are limited to a fixed number of turns, susceptible to sat…
Exploring Instruction Data Quality for Explainable Image Quality Assessment
Yunhao Li, Sijing Wu, Jun Jia +5
In recent years, with the rapid development of large multimodal models (LMMs), explainable image quality assessment (IQA) has attracted increasing attention, aiming to understand t…
Q-Mirror: Unlocking the Multi-Modal Potential of Scientific Text-Only QA Pairs
Junying Wang, Zicheng Zhang, Ye Shen +8
High-quality, multi-modal benchmarks are crucial for advancing scientific reasoning in large models yet their manual creation is costly and unscalable. To address this bottleneck,…
A Multi-To-One Interview Paradigm for Efficient MLLM Evaluation
Ye Shen, Junying Wang, Farong Wen +4
The rapid progress of Multi-Modal Large Language Models (MLLMs) has spurred the creation of numerous benchmarks. However, conventional full-coverage Question-Answering evaluations…
QoNext: Towards Next-generation QoE for Foundation Models
Yijin Guo, Farong Wen, Ye Shen +5
Existing evaluations of foundation models predominantly focus on output correctness, treating interaction as a static exchange of information. However, such perspectives overlook t…
Sycophancy under Pressure: Evaluating and Mitigating Sycophantic Bias via Adversarial Dialogues in Scientific QA
Kaiwei Zhang, Qi Jia, Zijian Chen +5
Large language models (LLMs), while increasingly used in domains requiring factual rigor, often display a troubling behavior: sycophancy, the tendency to align with user beliefs re…
User-centric Subjective Leaderboard by Customizable Reward Modeling
Qi Jia, Xiujie Song, Zicheng Zhang +4
Existing benchmarks for large language models (LLMs) predominantely focus on assessing their capabilities through verifiable tasks. Such objective and static benchmarks offer limit…