24 citations · 34 across the 18 of their papers we have counts for
14 papers · 1 filter
Qwen-AgentWorld: Language World Models for General Agents
Yuxin Zuo, Zikai Xiao, Li Sheng +30
A world model predicts environment dynamics based on current observations and actions, serving as a core cognitive mechanism for reasoning and planning. In this work, we investigat…
OccuBench: Evaluating AI Agents on Real-World Professional Tasks via Language Environment Simulation
Xiaomeng Hu, Yinger Zhang, Fei Huang +7
AI agents are expected to perform professional work across hundreds of occupational domains (from emergency department triage to nuclear reactor safety monitoring to customs import…
Outcome Accuracy is Not Enough: Aligning the Reasoning Process of Reward Models
Binghai Wang, Yantao Liu, Yuxuan Liu +13
Generative Reward Models (GenRMs) and LLM-as-a-Judge exhibit deceptive alignment by producing correct judgments for incorrect reasons, as they are trained and evaluated to prioriti…
Are Reasoning Models More Prone to Hallucination?
Zijun Yao, Yantao Liu, Yanxu Chen +5
Recently evolved large reasoning models (LRMs) show powerful performance in solving complex tasks with long chain-of-thought (CoT) reasoning capability. As these LRMs are mostly de…
Toward Generalizable Evaluation in the LLM Era: A Survey Beyond Benchmarks
Yixin Cao, Shibo Hong, Xinze Li +24
Large Language Models (LLMs) are advancing at an amazing speed and have become indispensable across academia, industry, and daily applications. To keep pace with the status quo, th…
PairJudge RM: Perform Best-of-N Sampling with Knockout Tournament
Yantao Liu, Zijun Yao, Rui Min +3
Best-of-N (BoN) sampling, a common strategy for test-time scaling of Large Language Models (LLMs), relies on reward models to select the best candidate solution from multiple gener…
RM-Bench: Benchmarking Reward Models of Language Models with Subtlety and Style
Yantao Liu, Zijun Yao, Rui Min +3
Reward models are critical in techniques like Reinforcement Learning from Human Feedback (RLHF) and Inference Scaling Laws, where they guide language model alignment and select opt…
Aligning Teacher with Student Preferences for Tailored Training Data Generation
Yantao Liu, Zhao Zhang, Zijun Yao +3
Large Language Models (LLMs) have shown significant promise as copilots in various tasks. Local deployment of LLMs on edge devices is necessary when handling privacy-sensitive data…
Untangle the KNOT: Interweaving Conflicting Knowledge and Reasoning Skills in Large Language Models
Yantao Liu, Zijun Yao, Xin Lv +5
Providing knowledge documents for large language models (LLMs) has emerged as a promising solution to update the static knowledge inherent in their parameters. However, knowledge i…
Evaluating Generative Language Models in Information Extraction as Subjective Question Correction
Yuchen Fan, Yantao Liu, Zijun Yao +3
Modern Large Language Models (LLMs) have showcased remarkable prowess in various tasks necessitating sophisticated cognitive behaviors. Nevertheless, a paradoxical performance disc…
An In-Context Schema Understanding Method for Knowledge Base Question Answering
Yantao Liu, Zixuan Li, Xiaolong Jin +5
The Knowledge Base Question Answering (KBQA) task aims to answer natural language questions based on a given knowledge base. Recently, Large Language Models (LLMs) have shown stron…
Nested Event Extraction upon Pivot Element Recogniton
Weicheng Ren, Zixuan Li, Xiaolong Jin +6
Nested Event Extraction (NEE) aims to extract complex event structures where an event contains other events as its arguments recursively. Nested events involve a kind of Pivot Elem…
KoRC: Knowledge oriented Reading Comprehension Benchmark for Deep Text Understanding
Zijun Yao, Yantao Liu, Xin Lv +4
Deep text understanding, which requires the connections between a given document and prior knowledge beyond its text, has been highlighted by many benchmarks in recent years. Howev…
KoLA: Carefully Benchmarking World Knowledge of Large Language Models
Jifan Yu, Xiaozhi Wang, Shangqing Tu +32
The unprecedented performance of large language models (LLMs) necessitates improvements in evaluations. Rather than merely exploring the breadth of LLM abilities, we believe meticu…