4 papers
Typological Alignment of Stack-Based Language Models on Mildly Context-Sensitive Artificial Languages
Nadine El-Naggar, Tatsuki Kuribayashi, Ted Briscoe
Some properties of languages, e.g., subject-object-verb (SOV) word order, are more prevalent than others among the thousands of attested natural languages (NLs). Such typological c…
What Kind of Language is Easy to Language-Model Under Curriculum Learning?
Nadine El-Naggar, Tatsuki Kuribayashi, Ted Briscoe
Many of the thousands of attested languages share common configurations of features, creating a spectrum from typologically very rare (e.g., object-verb-subject word order) or impo…
Which Word Orders Facilitate Length Generalization in LMs? An Investigation with GCG-Based Artificial Languages
Nadine El-Naggar, Tatsuki Kuribayashi, Ted Briscoe
Whether language models (LMs) have inductive biases that favor typologically frequent grammatical properties over rare, implausible ones has been investigated, typically using arti…
Emergent Word Order Universals from Cognitively-Motivated Language Models
Tatsuki Kuribayashi, Ryo Ueda, Ryo Yoshida +3
The world's languages exhibit certain so-called typological or implicational universals; for example, Subject-Object-Verb (SOV) languages typically use postpositions. Explaining th…