4 citations · 5 across the 21 of their papers we have counts for
16 papers · 1 filter
MME-CC: A Challenging Multi-Modal Evaluation Benchmark of Cognitive Capacity
Kaiyuan Zhang, Chenghao Yang, Zhoufutu Wen +19
As reasoning models scale rapidly, the essential role of multimodality in human cognition has come into sharp relief, driving a growing need to probe vision-centric cognitive behav…
DiscoX: Benchmarking Discourse-Level Translation task in Expert Domains
Xiying Zhao, Zhoufutu Wen, Zhixuan Chen +20
The evaluation of discourse-level translation in expert domains remains inadequate, despite its centrality to knowledge dissemination and cross-lingual scholarly communication. Whi…
COIG-Writer: A High-Quality Dataset for Chinese Creative Writing with Thought Processes
Yunwen Li, Shuangshuang Ying, Xingwei Qu +16
Large language models exhibit systematic deficiencies in creative writing, particularly in non-English contexts where training data is scarce and lacks process-level supervision. W…
Beyond Correctness: Evaluating Subjective Writing Preferences Across Cultures
Shuangshuang Ying, Yunwen Li, Xingwei Qu +21
Current preference learning methods achieve high accuracy on standard benchmarks but exhibit significant performance degradation when objective quality signals are removed. We intr…
FinSearchComp: Towards a Realistic, Expert-Level Evaluation of Financial Search and Reasoning
Liang Hu, Jianpeng Jiao, Jiashuo Liu +20
Search has emerged as core infrastructure for LLM-based agents and is widely viewed as critical on the path toward more general intelligence. Finance is a particularly demanding pr…
FutureX: An Advanced Live Benchmark for LLM Agents in Future Prediction
Zhiyuan Zeng, Jiashuo Liu, Siyuan Chen +28
Future prediction is a complex task for LLM agents, requiring a high level of analytical thinking, information gathering, contextual understanding, and decision-making under uncert…
TreePO: Bridging the Gap of Policy Optimization and Efficacy and Inference Efficiency with Heuristic Tree-based Modeling
Yizhi Li, Qingshui Gu, Zhoufutu Wen +14
Recent advancements in aligning large language models via reinforcement learning have achieved remarkable gains in solving complex reasoning problems, but at the cost of expensive…
First Return, Entropy-Eliciting Explore
Tianyu Zheng, Tianshun Xing, Qingshui Gu +10
Reinforcement Learning from Verifiable Rewards (RLVR) improves the reasoning abilities of Large Language Models (LLMs) but it struggles with unstable exploration. We propose FR3E (…
SciDA: Scientific Dynamic Assessor of LLMs
Junting Zhou, Tingjia Miao, Yiyan Liao +15
Advancement in Large Language Models (LLMs) reasoning capabilities enables them to solve scientific problems with enhanced efficacy. Thereby, a high-quality benchmark for comprehen…
KORGym: A Dynamic Game Platform for LLM Reasoning Evaluation
Jiajun Shi, Jian Yang, Jiaheng Liu +26
Recent advancements in large language models (LLMs) underscore the need for more comprehensive evaluation methods to accurately assess their reasoning capabilities. Existing benchm…
MARS-Bench: A Multi-turn Athletic Real-world Scenario Benchmark for Dialogue Evaluation
Chenghao Yang, Yinbo Luo, Zhoufutu Wen +8
Large Language Models (\textbf{LLMs}), e.g. ChatGPT, have been widely adopted in real-world dialogue applications. However, LLMs' robustness, especially in handling long complex di…
IV-Bench: A Benchmark for Image-Grounded Video Perception and Reasoning in Multimodal LLMs
David Ma, Yuanxing Zhang, Jincheng Ren +17
Existing evaluation frameworks for Multimodal Large Language Models (MLLMs) primarily focus on image reasoning or general video understanding tasks, largely overlooking the signifi…
SuperGPQA: Scaling LLM Evaluation across 285 Graduate Disciplines
P Team, Xinrun Du, Yifan Yao +94
Large language models (LLMs) have demonstrated remarkable proficiency in mainstream academic disciplines such as mathematics, physics, and computer science. However, human knowledg…
CryptoX : Compositional Reasoning Evaluation of Large Language Models
Jiajun Shi, Chaoren Wei, Liqun Yang +7
The compositional reasoning capacity has long been regarded as critical to the generalization and intelligence emergence of large language models LLMs. However, despite numerous re…
SimpleVQA: Multimodal Factuality Evaluation for Multimodal Large Language Models
Xianfu Cheng, Wei Zhang, Shiwei Zhang +16
The increasing application of multi-modal large language models (MLLMs) across various sectors have spotlighted the essence of their output reliability and accuracy, particularly t…
Quantification of Large Language Model Distillation
Sunbowen Lee, Junting Zhou, Chang Ao +11
Model distillation is a fundamental technique in building large language models (LLMs), transferring knowledge from a teacher model to a student model. However, distillation can le…