2 citations · 2 across the 4 of their papers we have counts for
5 papers
Typological Alignment of Stack-Based Language Models on Mildly Context-Sensitive Artificial Languages
Nadine El-Naggar, Tatsuki Kuribayashi, Ted Briscoe
Some properties of languages, e.g., subject-object-verb (SOV) word order, are more prevalent than others among the thousands of attested natural languages (NLs). Such typological c…
What Kind of Language is Easy to Language-Model Under Curriculum Learning?
Nadine El-Naggar, Tatsuki Kuribayashi, Ted Briscoe
Many of the thousands of attested languages share common configurations of features, creating a spectrum from typologically very rare (e.g., object-verb-subject word order) or impo…
Which Word Orders Facilitate Length Generalization in LMs? An Investigation with GCG-Based Artificial Languages
Nadine El-Naggar, Tatsuki Kuribayashi, Ted Briscoe
Whether language models (LMs) have inductive biases that favor typologically frequent grammatical properties over rare, implausible ones has been investigated, typically using arti…
Theoretical Conditions and Empirical Failure of Bracket Counting on Long Sequences with Linear Recurrent Networks
Nadine El-Naggar, Pranava Madhyastha, Tillman Weyde
Previous work has established that RNNs with an unbounded activation function have the capacity to count exactly. However, it has also been shown that RNNs are challenging to train…
Exploring the Long-Term Generalization of Counting Behavior in RNNs
Nadine El-Naggar, Pranava Madhyastha, Tillman Weyde
In this study, we investigate the generalization of LSTM, ReLU and GRU models on counting tasks over long sequences. Previous theoretical work has established that RNNs with ReLU a…