111 citations · 462 across the 65 of their papers we have counts for
77 papers
Unsupervised lexicon learning from speech is limited by representations rather than clustering
Danel Slabbert, Simon Malan, Herman Kamper
Zero-resource word segmentation and clustering systems aim to tokenise speech into word-like units without access to text labels. Despite progress, the induced lexicons are still f…
Should Top-Down Clustering Affect Boundaries in Unsupervised Word Discovery?
Simon Malan, Benjamin van Niekerk, Herman Kamper
We investigate the problem of segmenting unlabeled speech into word-like units and clustering these to create a lexicon. Prior work can be categorized into two frameworks. Bottom-u…
Analyzing and Improving Speaker Similarity Assessment for Speech Synthesis
Marc-André Carbonneau, Benjamin van Niekerk, Hugo Seuté +3
Modeling voice identity is challenging due to its multifaceted nature. In generative speech systems, identity is often assessed using automatic speaker verification (ASV) embedding…
LinearVC: Linear transformations of self-supervised features through the lens of voice conversion
Herman Kamper, Benjamin van Niekerk, Julian Zaïdi +1
We introduce LinearVC, a simple voice conversion method that sheds light on the structure of self-supervised representations. First, we show that simple linear transformations of s…
Unsupervised Word Discovery: Boundary Detection with Clustering vs. Dynamic Programming
Simon Malan, Benjamin van Niekerk, Herman Kamper
We look at the long-standing problem of segmenting unlabeled speech into word-like segments and clustering these into a lexicon. Several previous methods use a scoring model couple…
Spoken-Term Discovery using Discrete Speech Units
Benjamin van Niekerk, Julian Zaïdi, Marc-André Carbonneau +1
Discovering a lexicon from unlabeled audio is a longstanding challenge for zero-resource speech processing. One approach is to search for frequently occurring patterns in speech. W…
Visually Grounded Speech Models have a Mutual Exclusivity Bias
Leanne Nortje, Dan Oneaţă, Yevgen Matusevych +1
When children learn new words, they employ constraints such as the mutual exclusivity (ME) bias: a novel word is mapped to a novel object rather than a familiar one. This bias has…
Revisiting speech segmentation and lexicon learning with better features
Herman Kamper, Benjamin van Niekerk
We revisit a self-supervised method that segments unlabelled speech into word-like segments. We start from the two-stage duration-penalised dynamic programming method that performs…
Voice Conversion for Stuttered Speech, Instruments, Unseen Languages and Textually Described Voices
Matthew Baas, Herman Kamper
Voice conversion aims to convert source speech into a target voice using recordings of the target speaker as a reference. Newer models are producing increasingly realistic output.…
Rhythm Modeling for Voice Conversion
Benjamin van Niekerk, Marc-André Carbonneau, Herman Kamper
Voice conversion aims to transform source speech into a different target voice. However, typical voice conversion systems do not account for rhythm, which is an important factor in…
Leveraging multilingual transfer for unsupervised semantic acoustic word embeddings
Christiaan Jacobs, Herman Kamper
Acoustic word embeddings (AWEs) are fixed-dimensional vector representations of speech segments that encode phonetic content so that different realisations of the same word have si…
Disentanglement in a GAN for Unconditional Speech Synthesis
Matthew Baas, Herman Kamper
Can we develop a model that can synthesize realistic speech directly from a latent space, without explicit conditioning? Despite several efforts over the last decade, previous adve…
Visually grounded few-shot word learning in low-resource settings
Leanne Nortje, Dan Oneata, Herman Kamper
We propose a visually grounded speech model that learns new words and their visual depictions from just a few word-image example pairs. Given a set of test images and a spoken quer…
Towards hate speech detection in low-resource languages: Comparing ASR to acoustic word embeddings on Wolof and Swahili
Christiaan Jacobs, Nathanaël Carraz Rakotonirina, Everlyn Asiko Chimoto +2
We consider hate speech detection through keyword spotting on radio broadcasts. One approach is to build an automatic speech recognition (ASR) system for the target low-resource la…
Voice Conversion With Just Nearest Neighbors
Matthew Baas, Benjamin van Niekerk, Herman Kamper
Any-to-any voice conversion aims to transform source speech into a target voice with just a few examples of the target speaker as a reference. Recent methods produce convincing con…
Visually grounded few-shot word acquisition with fewer shots
Leanne Nortje, Benjamin van Niekerk, Herman Kamper
We propose a visually grounded speech model that acquires new words and their visual depictions from just a few word-image example pairs. Given a set of test images and a spoken qu…
Mitigating Catastrophic Forgetting for Few-Shot Spoken Word Classification Through Meta-Learning
Ruan van der Merwe, Herman Kamper
We consider the problem of few-shot spoken word classification in a setting where a model is incrementally introduced to new word classes. This would occur in a user-defined keywor…
TransFusion: Transcribing Speech with Multinomial Diffusion
Matthew Baas, Kevin Eloff, Herman Kamper
Diffusion models have shown exceptional scaling properties in the image synthesis domain, and initial attempts have shown similar benefits for applying diffusion to unconditional t…
Towards visually prompted keyword localisation for zero-resource spoken languages
Leanne Nortje, Herman Kamper
Imagine being able to show a system a visual depiction of a keyword and finding spoken utterances that contain this keyword from a zero-resource speech corpus. We formalise this ta…
YFACC: A Yorùbá speech-image dataset for cross-lingual keyword localisation through visual grounding
Kayode Olaleye, Dan Oneata, Herman Kamper
Visually grounded speech (VGS) models are trained on images paired with unlabelled spoken captions. Such models could be used to build speech systems in settings where it is imposs…
GAN You Hear Me? Reclaiming Unconditional Speech Synthesis from Diffusion Models
Matthew Baas, Herman Kamper
We propose AudioStyleGAN (ASGAN), a new generative adversarial network (GAN) for unconditional speech synthesis. As in the StyleGAN family of image synthesis models, ASGAN maps sam…
A Temporal Extension of Latent Dirichlet Allocation for Unsupervised Acoustic Unit Discovery
Werner van der Merwe, Herman Kamper, Johan du Preez
Latent Dirichlet allocation (LDA) is widely used for unsupervised topic modelling on sets of documents. No temporal information is used in the model. However, there is often a rela…
Keyword localisation in untranscribed speech using visually grounded speech models
Kayode Olaleye, Dan Oneata, Herman Kamper
Keyword localisation is the task of finding where in a speech utterance a given query keyword occurs. We investigate to what extent keyword localisation is possible using a visuall…
Word Segmentation on Discovered Phone Units with Dynamic Programming and Self-Supervised Scoring
Herman Kamper
Recent work on unsupervised speech segmentation has used self-supervised models with phone and word segmentation modules that are trained jointly. This paper instead revisits an ol…