51 citations · 116 across the 7 of their papers we have counts for
8 papers
Modelling word learning and recognition using visually grounded speech
Danny Merkx, Sebastiaan Scholten, Stefan L. Frank +2
Background: Computational models of speech recognition often assume that the set of target words is already given. This implies that these models do not learn to recognise speech f…
Seeing the advantage: visually grounding word embeddings to better capture human semantic knowledge
Danny Merkx, Stefan L. Frank, Mirjam Ernestus
Distributional semantic models capture word-level meaning that is useful in many natural language processing tasks and have even been shown to capture cognitive aspects of word mea…
Semantic sentence similarity: size does not always matter
Danny Merkx, Stefan L. Frank, Mirjam Ernestus
This study addresses the question whether visually grounded speech recognition (VGS) models learn to capture sentence semantics without access to any prior linguistic knowledge. We…
Learning to Recognise Words using Visually Grounded Speech
Sebastiaan Scholten, Danny Merkx, Odette Scharenborg
We investigated word recognition in a Visually Grounded Speech model. The model has been trained on pairs of images and spoken captions to create visually grounded embeddings which…
Human Sentence Processing: Recurrence or Attention?
Danny Merkx, Stefan L. Frank
Recurrent neural networks (RNNs) have long been an architecture of interest for computational models of human sentence processing. The recently introduced Transformer architecture…
Language learning using Speech to Image retrieval
Danny Merkx, Stefan L. Frank, Mirjam Ernestus
Humans learn language by interaction with their environment and listening to other humans. It should also be possible for computational models to learn language directly from speec…
Learning semantic sentence representations from visually grounded language without lexical knowledge
Danny Merkx, Stefan Frank
Current approaches to learning semantic representations of sentences often use prior word-level knowledge. The current study aims to leverage visual information in order to capture…
Linguistic unit discovery from multi-modal inputs in unwritten languages: Summary of the "Speaking Rosetta" JSALT 2017 Workshop
Odette Scharenborg, Laurent Besacier, Alan Black +16
We summarize the accomplishments of a multi-disciplinary workshop exploring the computational and scientific issues surrounding the discovery of linguistic units (subwords and word…