565 citations · 818 across the 51 of their papers we have counts for
8 papers · 1 filter
Location Attention for Extrapolation to Longer Sequences
Yann Dubois, Gautier Dagan, Dieuwke Hupkes +1
Neural networks are surprisingly good at interpolating and perform remarkably well when the training set examples resemble those in the test set. However, they are often unable to…
Analysing Neural Language Models: Contextual Decomposition Reveals Default Reasoning in Number and Gender Assignment
Jaap Jumelet, Willem Zuidema, Dieuwke Hupkes
Extensive research has recently shown that recurrent neural language models are able to process a wide range of grammatical phenomena. How these models are able to perform these re…
Compositionality decomposed: how do neural networks generalise?
Dieuwke Hupkes, Verna Dankers, Mathijs Mul +1
Despite a multitude of empirical studies, little consensus exists on whether neural networks are able to generalise compositionally, a controversy that, in part, stems from a lack…
Assessing incrementality in sequence-to-sequence models
Dennis Ulmer, Dieuwke Hupkes, Elia Bruni
Since their inception, encoder-decoder models have successfully been applied to a wide array of problems in computational linguistics. The most recent successes are predominantly d…
On the Realization of Compositionality in Neural Networks
Joris Baan, Jana Leible, Mitja Nikolaus +5
We present a detailed comparison of two types of sequence to sequence models trained to conduct a compositional task. The models are architecturally identical at inference time, bu…
Transcoding compositionally: using attention to find more generalizable solutions
Kris Korrel, Dieuwke Hupkes, Verna Dankers +1
While sequence-to-sequence models have shown remarkable generalization power across several natural language tasks, their construct of solutions are argued to be less compositional…
The emergence of number and syntax units in LSTM language models
Yair Lakretz, German Kruszewski, Theo Desbordes +3
Recent work has shown that LSTMs trained on a generic language modeling objective capture syntax-sensitive generalizations such as long-distance number agreement. We have however n…
Formal models of Structure Building in Music, Language and Animal Songs
Willem Zuidema, Dieuwke Hupkes, Geraint Wiggins +2
Human language, music and a variety of animal vocalisations constitute ways of sonic communication that exhibit remarkable structural complexity. While the complexities of language…