51 citations · 165 across the 7 of their papers we have counts for
7 papers
Modelling word learning and recognition using visually grounded speech
Danny Merkx, Sebastiaan Scholten, Stefan L. Frank +2
Background: Computational models of speech recognition often assume that the set of target words is already given. This implies that these models do not learn to recognise speech f…
Seeing the advantage: visually grounding word embeddings to better capture human semantic knowledge
Danny Merkx, Stefan L. Frank, Mirjam Ernestus
Distributional semantic models capture word-level meaning that is useful in many natural language processing tasks and have even been shown to capture cognitive aspects of word mea…
Semantic sentence similarity: size does not always matter
Danny Merkx, Stefan L. Frank, Mirjam Ernestus
This study addresses the question whether visually grounded speech recognition (VGS) models learn to capture sentence semantics without access to any prior linguistic knowledge. We…
Human Sentence Processing: Recurrence or Attention?
Danny Merkx, Stefan L. Frank
Recurrent neural networks (RNNs) have long been an architecture of interest for computational models of human sentence processing. The recently introduced Transformer architecture…
Language learning using Speech to Image retrieval
Danny Merkx, Stefan L. Frank, Mirjam Ernestus
Humans learn language by interaction with their environment and listening to other humans. It should also be possible for computational models to learn language directly from speec…
Learning semantic sentence representations from visually grounded language without lexical knowledge
Danny Merkx, Stefan Frank
Current approaches to learning semantic representations of sentences often use prior word-level knowledge. The current study aims to leverage visual information in order to capture…
Lexical representation explains cortical entrainment during speech comprehension
Stefan Frank, Jinbiao Yang
Results from a recent neuroimaging study on spoken sentence comprehension have been interpreted as evidence for cortical entrainment to hierarchical syntactic structure. We present…