31 citations · 31 across the 5 of their papers we have counts for
6 papers · 1 filter
A Systematic Analysis of the Predictive Power of LM Surprisal in Reading Chinese
Hongao Zhu, Muxiaoqiao Xu, Yikang Liu +4
This study analyzes the predictive power of LM-derived, token-level surprisal on Mandarin Chinese reading times. We first propose the Shortest Matching Sequence (SMS), an alignment…
Math Natural Language Inference: this should be easy!
Valeria de Paiva, Qiyue Gao, Hai Hu +4
We ask whether contemporary LLMs are able to perform natural language inference (NLI) tasks on mathematical texts. We call this the Math NLI problem. We construct a corpus of Math…
Translationese-index: Using Likelihood Ratios for Graded and Generalizable Measurement of Translationese
Yikang Liu, Wanyang Zhang, Yiming Wang +6
Translationese refers to linguistic properties that usually occur in translated texts. Previous works study translationese by framing it as a binary classification between original…
A Systematic Assessment of Language Models with Linguistic Minimal Pairs in Chinese
Yikang Liu, Yeting Shen, Hongao Zhu +9
We present ZhoBLiMP, the largest linguistic minimal pair benchmark for Chinese, with over 100 paradigms, ranging from topicalization to the \textit{Ba} construction. We then train…
MELA: Multilingual Evaluation of Linguistic Acceptability
Ziyin Zhang, Yikang Liu, Weifang Huang +3
In this work, we present the largest benchmark to date on linguistic acceptability: Multilingual Evaluation of Linguistic Acceptability -- MELA, with 46K samples covering 10 langua…
ArguGPT: evaluating, understanding and identifying argumentative essays generated by GPT models
Yikang Liu, Ziyin Zhang, Wanyang Zhang +5
AI generated content (AIGC) presents considerable challenge to educators around the world. Instructors need to be able to detect such text generated by large language models, eithe…