2 citations · 3 across the 10 of their papers we have counts for
11 papers
A Systematic Analysis of the Predictive Power of LM Surprisal in Reading Chinese
Hongao Zhu, Muxiaoqiao Xu, Yikang Liu +4
This study analyzes the predictive power of LM-derived, token-level surprisal on Mandarin Chinese reading times. Using a suite of Chinese-Pythia models (14M--1.4B) trained from scr…
The First ChineseBabyLM Challenge: training data-efficient and cognitively plausible language models for Chinese
Siyuan Song, Zhiheng Qian, Yunhao Zhang +11
This paper presents the first ChineseBabyLM Challenge, organized as part of NLPCC 2026. The challenge asked participants to train language models from scratch using no more than 10…
Reasoning or Memorization: Can LLMs Understand and Generate Chinese Xiehouyu Riddles?
Hai Hu, Siyuan Song, Chongtian Shao +3
In this paper, we push the boundary of LLM reasoning by testing them in a Chinese language game, xiehouyu, with novel xiehouyu created by linguists that had not existed before to a…
Bears, all bears, and some bears. Language Constraints on Language Models' Inductive Inferences
Sriram Padmanabhan, Siyuan Song, Kanishka Misra
Language places subtle constraints on how we make inductive inferences. Developmental evidence by Gelman et al. (2002) has shown children (4 years and older) to differentiate among…
What Can String Probability Tell Us About Grammaticality?
Jennifer Hu, Ethan Gotlieb Wilcox, Siyuan Song +2
What have language models (LMs) learned about grammar? This question remains hotly debated, with major ramifications for linguistic theory. However, since probability and grammatic…
BabyBabelLM: A Multilingual Benchmark of Developmentally Plausible Training Data
Jaap Jumelet, Abdellah Fourtassi, Akari Haga +23
We present BabyBabelLM, a multilingual collection of datasets modeling the language a person observes from birth until they acquire a native language. We curate developmentally pla…
Global PIQA: Evaluating Commonsense Reasoning Across 100+ Languages and Cultures
Tyler A. Chang, Catherine Arnett, Abdelrahman Sadallah +377
To date, there exist almost no culturally-specific evaluation benchmarks for large language models (LLMs) that cover a large number of languages and cultures. In this paper, we pre…
Privileged Self-Access Matters for Introspection in AI
Siyuan Song, Harvey Lederman, Jennifer Hu +1
Whether AI models can introspect is an increasingly important practical question. But there is no consensus on how introspection is to be defined. Beginning from a recently propose…
Language Models Fail to Introspect About Their Knowledge of Language
Siyuan Song, Jennifer Hu, Kyle Mahowald
There has been recent interest in whether large language models (LLMs) can introspect about their own internal states. Such abilities would make LLMs more interpretable, and also v…
A Systematic Assessment of Language Models with Linguistic Minimal Pairs in Chinese
Yikang Liu, Yeting Shen, Hongao Zhu +9
We present ZhoBLiMP, the largest linguistic minimal pair benchmark for Chinese, with over 100 paradigms, ranging from topicalization to the \textit{Ba} construction. We then train…
Do Large Language Models Understand Conversational Implicature -- A case study with a chinese sitcom
Shisen Yue, Siyuan Song, Xinyuan Cheng +1
Understanding the non-literal meaning of an utterance is critical for large language models (LLMs) to become human-like social communicators. In this work, we introduce SwordsmanIm…