8 papers
BabyLM's First Constructions: Causal probing provides a signal of learning
Joshua Rozner, Leonie Weissweiler, Cory Shain
Construction grammar posits that language learners acquire constructions (form-meaning pairings) from the statistics of their environment. Recent work supports this hypothesis by s…
MultiBLiMP 1.0: A Massively Multilingual Benchmark of Linguistic Minimal Pairs
Jaap Jumelet, Leonie Weissweiler, Joakim Nivre +1
We introduce MultiBLiMP 1.0, a massively multilingual benchmark of linguistic minimal pairs, covering 101 languages and 2 types of subject-verb agreement, containing more than 128,…
Both Direct and Indirect Evidence Contribute to Dative Alternation Preferences in Language Models
Qing Yao, Kanishka Misra, Leonie Weissweiler +1
Language models (LMs) tend to show human-like preferences on a number of syntactic phenomena, but the extent to which these are attributable to direct exposure to the phenomena or…
Constructions are Revealed in Word Distributions
Joshua Rozner, Leonie Weissweiler, Kyle Mahowald +1
Construction grammar posits that constructions, or form-meaning pairings, are acquired through experience with language (the distributional learning hypothesis). But how much infor…
Linguistic Generalizations are not Rules: Impacts on Evaluation of LMs
Leonie Weissweiler, Kyle Mahowald, Adele Goldberg
Linguistic evaluations of how well LMs generalize to produce or understand language often implicitly take for granted that natural languages are generated by symbolic rules. Accord…
Derivational Morphology Reveals Analogical Generalization in Large Language Models
Valentin Hofmann, Leonie Weissweiler, David Mortensen +2
What mechanisms underlie linguistic generalization in large language models (LLMs)? This question has attracted considerable attention, with most studies analyzing the extent to wh…
Models Can and Should Embrace the Communicative Nature of Human-Generated Math
Sasha Boguraev, Ben Lipkin, Leonie Weissweiler +1
Math is constructed by people for people: just as natural language corpora reflect not just propositions but the communicative goals of language users, the math data that models ar…
SYNTHEVAL: Hybrid Behavioral Testing of NLP Models with Synthetic CheckLists
Raoyuan Zhao, Abdullatif Köksal, Yihong Liu +3
Traditional benchmarking in NLP typically involves using static held-out test sets. However, this approach often results in an overestimation of performance and lacks the ability t…