5 papers
Anything Goes? A Crosslinguistic Study of (Im)possible Language Learning in LMs
Xiulin Yang, Tatsuya Aoyama, Yuekun Yao +1
Do language models (LMs) offer insights into human language learning? A common argument against this idea is that because their architecture and training paradigm are so vastly dif…
Global PIQA: Evaluating Commonsense Reasoning Across 100+ Languages and Cultures
Tyler A. Chang, Catherine Arnett, Abdelrahman Sadallah +377
To date, there exist almost no culturally-specific evaluation benchmarks for large language models (LLMs) that cover a large number of languages and cultures. In this paper, we pre…
Unpacking Let Alone: Human-Scale Models Generalize to a Rare Construction in Form but not Meaning
Wesley Scivetti, Tatsuya Aoyama, Ethan Wilcox +1
Humans have a remarkable ability to acquire and understand grammatical phenomena that are seen rarely, if ever, during childhood. Recent evidence suggests that language models with…
Language Models Grow Less Humanlike beyond Phase Transition
Tatsuya Aoyama, Ethan Wilcox
LMs' alignment with human reading behavior (i.e. psychometric predictive power; PPP) is known to improve during pretraining up to a tipping point, beyond which it either plateaus o…
Identifying Fairness Issues in Automatically Generated Testing Content
Kevin Stowe, Benny Longwill, Alyssa Francis +3
Natural language generation tools are powerful and effective for generating content. However, language models are known to display bias and fairness issues, making them impractical…