3 papers
cs.CL2024
Visually Grounded Speech Models for Low-resource Languages and Cognitive Modelling
Leanne Nortje
This dissertation examines visually grounded speech (VGS) models that learn from unlabelled speech paired with images. It focuses on applications for low-resource languages and und…
cs.CL2024
Visually Grounded Speech Models have a Mutual Exclusivity Bias
Leanne Nortje, Dan Oneaţă, Yevgen Matusevych +1
When children learn new words, they employ constraints such as the mutual exclusivity (ME) bias: a novel word is mapped to a novel object rather than a familiar one. This bias has…
cs.CL2023
Visually grounded few-shot word acquisition with fewer shots
Leanne Nortje, Benjamin van Niekerk, Herman Kamper
We propose a visually grounded speech model that acquires new words and their visual depictions from just a few word-image example pairs. Given a set of test images and a spoken qu…