8 papers · 1 filter
Optimizing Diversity and Quality through Base-Aligned Model Collaboration
Yichen Wang, Chenghao Yang, Tenghao Huang +3
Alignment has greatly improved large language models (LLMs)' output quality at the cost of diversity, yielding highly similar outputs across generations, especially in open-ended g…
The Chameleon's Limit: Investigating Persona Collapse and Homogenization in Large Language Models
Yunze Xiao, Vivienne J. Zhang, Chenghao Yang +3
Applications based on large language models (LLMs), such as multi-agent simulations, require population diversity among agents. We identify a pervasive failure mode we term \emph{P…
WorldTravel: A Realistic Multimodal Travel-Planning Benchmark with Tightly Coupled Constraints
Zexuan Wang, Chenghao Yang, Yingqi Que +18
Real-world autonomous planning requires coordinating tightly coupled constraints where a single decision dictates the feasibility of all subsequent actions. However, existing bench…
MME-CC: A Challenging Multi-Modal Evaluation Benchmark of Cognitive Capacity
Kaiyuan Zhang, Chenghao Yang, Zhoufutu Wen +19
As reasoning models scale rapidly, the essential role of multimodality in human cognition has come into sharp relief, driving a growing need to probe vision-centric cognitive behav…
DiscoX: Benchmarking Discourse-Level Translation task in Expert Domains
Xiying Zhao, Zhoufutu Wen, Zhixuan Chen +20
The evaluation of discourse-level translation in expert domains remains inadequate, despite its centrality to knowledge dissemination and cross-lingual scholarly communication. Whi…
MARS-Bench: A Multi-turn Athletic Real-world Scenario Benchmark for Dialogue Evaluation
Chenghao Yang, Yinbo Luo, Zhoufutu Wen +8
Large Language Models (\textbf{LLMs}), e.g. ChatGPT, have been widely adopted in real-world dialogue applications. However, LLMs' robustness, especially in handling long complex di…