15 citations · 32 across the 6 of their papers we have counts for
7 papers
Towards Developmentally Plausible Rewards: Communicative Success as a Learning Signal for Interactive Language Models
Lennart Stöpler, Rufat Asadli, Mitja Nikolaus +2
We propose a method for training language models in an interactive setting inspired by child language acquisition. In our setting, a speaker attempts to communicate some informatio…
Automatic Annotation of Grammaticality in Child-Caregiver Conversations
Mitja Nikolaus, Abhishek Agrawal, Petros Kaklamanis +2
The acquisition of grammar has been a central question to adjudicate between theories of language acquisition. In order to conduct faster, more reproducible, and larger-scale corpu…
Modality-Agnostic fMRI Decoding of Vision and Language
Mitja Nikolaus, Milad Mozafari, Nicholas Asher +2
Previous studies have shown that it is possible to map brain activation data of subjects viewing images onto the feature representation space of not only vision models (modality-sp…
Do Vision-and-Language Transformers Learn Grounded Predicate-Noun Dependencies?
Mitja Nikolaus, Emmanuelle Salin, Stephane Ayache +2
Recent advances in vision-and-language modeling have seen the development of Transformer architectures that achieve remarkable performance on multimodal reasoning tasks. Yet, the e…
Learning English with Peppa Pig
Mitja Nikolaus, Afra Alishahi, Grzegorz Chrupała
Recent computational models of the acquisition of spoken language via grounding in perception exploit associations between the spoken and visual modalities and learn to represent s…
Compositional Generalization in Image Captioning
Mitja Nikolaus, Mostafa Abdou, Matthew Lamm +2
Image captioning models are usually evaluated on their ability to describe a held-out set of images, not on their ability to generalize to unseen concepts. We study the problem of…
On the Realization of Compositionality in Neural Networks
Joris Baan, Jana Leible, Mitja Nikolaus +5
We present a detailed comparison of two types of sequence to sequence models trained to conduct a compositional task. The models are architecturally identical at inference time, bu…