6 papers · 1 filter
Asking What a Director Sees Can Change What Vision-Language Models Choose
Jiaxuan Guo, Xingyu Ren, Kejia Zhang +3
Vision-language models are now routinely tested on theory of mind: whether they can work out what another person sees, knows, and means. A common design asks the model to report th…
Scoring the Wrong Question: Readout Failures in Constrained-Option Evaluation
Jiaxuan Guo, Kejia Zhang, Shuo Xin +3
Constrained-option scoring reads a model's probabilities for a fixed set of permitted answers, so it returns a score even when the model is about to write something else. We study…
Cross-Task Dissociation in Frontier Vision-Language Model Theory of Mind
Kejia Zhang, Youran Sun, Chugang Yi +1
Do frontier vision-language models present a coherent Theory-of-Mind (ToM) profile across tasks, matching the same human reference group, or does that profile fragment from one par…
Reasoning or Memorization: Can LLMs Understand and Generate Chinese Xiehouyu Riddles?
Hai Hu, Siyuan Song, Chongtian Shao +3
In this paper, we push the boundary of LLM reasoning by testing them in a Chinese language game, xiehouyu, with novel xiehouyu created by linguists that had not existed before to a…
PerspectiveGap: A Benchmark for Multi-Agent Orchestration Prompting
Youran Sun, Xingyu Ren, Kejia Zhang +2
Real-world LLM applications are moving beyond single-agent workflows toward orchestrated multi-agent systems, yet current models still struggle to determine what each sub-agent nee…
A Systematic Assessment of Language Models with Linguistic Minimal Pairs in Chinese
Yikang Liu, Yeting Shen, Hongao Zhu +9
We present ZhoBLiMP, the largest linguistic minimal pair benchmark for Chinese, with over 100 paradigms, ranging from topicalization to the \textit{Ba} construction. We then train…