1 citations · 1 across the 11 of their papers we have counts for
12 papers
The mutual exclusivity bias of bilingual visually grounded speech models
Dan Oneata, Leanne Nortje, Yevgen Matusevych +1
Mutual exclusivity (ME) is a strategy where a novel word is associated with a novel object rather than a familiar one, facilitating language learning in children. Recent work has f…
Visually Grounded Speech Models for Low-resource Languages and Cognitive Modelling
Leanne Nortje
This dissertation examines visually grounded speech (VGS) models that learn from unlabelled speech paired with images. It focuses on applications for low-resource languages and und…
Improved Visually Prompted Keyword Localisation in Real Low-Resource Settings
Leanne Nortje, Dan Oneata, Gabriel Pirlogeanu +1
Given an image query, visually prompted keyword localisation (VPKL) aims to find occurrences of the depicted word in a speech collection. This can be useful when transcriptions are…
Visually Grounded Speech Models have a Mutual Exclusivity Bias
Leanne Nortje, Dan Oneaţă, Yevgen Matusevych +1
When children learn new words, they employ constraints such as the mutual exclusivity (ME) bias: a novel word is mapped to a novel object rather than a familiar one. This bias has…
Visually grounded few-shot word learning in low-resource settings
Leanne Nortje, Dan Oneata, Herman Kamper
We propose a visually grounded speech model that learns new words and their visual depictions from just a few word-image example pairs. Given a set of test images and a spoken quer…
Visually grounded few-shot word acquisition with fewer shots
Leanne Nortje, Benjamin van Niekerk, Herman Kamper
We propose a visually grounded speech model that acquires new words and their visual depictions from just a few word-image example pairs. Given a set of test images and a spoken qu…
Towards visually prompted keyword localisation for zero-resource spoken languages
Leanne Nortje, Herman Kamper
Imagine being able to show a system a visual depiction of a keyword and finding spoken utterances that contain this keyword from a zero-resource speech corpus. We formalise this ta…
Analyzing Speaker Information in Self-Supervised Models to Improve Zero-Resource Speech Processing
Benjamin van Niekerk, Leanne Nortje, Matthew Baas +1
Contrastive predictive coding (CPC) aims to learn representations of speech by distinguishing future observations from a set of negative examples. Previous work has shown that line…
Direct multimodal few-shot learning of speech and images
Leanne Nortje, Herman Kamper
We propose direct multimodal few-shot models that learn a shared embedding space of spoken words and images from only a few paired examples. Imagine an agent is shown an image alon…
Unsupervised vs. transfer learning for multimodal one-shot matching of speech and images
Leanne Nortje, Herman Kamper
We consider the task of multimodal one-shot speech-image matching. An agent is shown a picture along with a spoken word describing the object in the picture, e.g. cookie, broccoli…
Vector-quantized neural networks for acoustic unit discovery in the ZeroSpeech 2020 challenge
Benjamin van Niekerk, Leanne Nortje, Herman Kamper
In this paper, we explore vector quantization for acoustic unit discovery. Leveraging unlabelled data, we aim to learn discrete representations of speech that separate phonetic con…
Unsupervised acoustic unit discovery for speech synthesis using discrete latent-variable neural networks
Ryan Eloff, André Nortje, Benjamin van Niekerk +7
For our submission to the ZeroSpeech 2019 challenge, we apply discrete latent-variable neural networks to unlabelled speech and use the discovered units for speech synthesis. Unsup…