Unsupervised word segmentation and lexicon discovery using acoustic word embeddings

Abstract

In settings where only unlabelled speech data is available, speech technology needs to be developed without transcriptions, pronunciation dictionaries, or language modelling text. A similar problem is faced when modelling infant language acquisition. In these cases, categorical linguistic structure needs to be discovered directly from speech audio. We present a novel unsupervised Bayesian model that segments unlabelled speech and clusters the segments into hypothesized word groupings. The result is a complete unsupervised tokenization of the input speech in terms of discovered word types. In our approach, a potential word segment (of arbitrary length) is embedded in a fixed-dimensional acoustic vector space. The model, implemented as a Gibbs sampler, then builds a whole-word acoustic model in this space while jointly performing segmentation. We report word error rates in a small-vocabulary connected digit recognition task by mapping the unsupervised decoded output to ground truth transcriptions. The model achieves around 20% error rate, outperforming a previous HMM-based system by about 10% absolute. Moreover, in contrast to the baseline, our model does not require a pre-specified vocabulary size.

arXiv comment11 pages, 8 figures; Accepted to the IEEE/ACM Transactions on Audio, Speech, and Language Processing

References in corpus (2)

• An Efficient, Probabilistically Sound Algorithm for Segmentation and Word Discovery
• Deep convolutional acoustic word embeddings using word-pair side information

Cited by in corpus (15)

• A segmental framework for fully-unsupervised large-vocabulary speech recognition
• Unsupervised Cross-Modal Alignment of Speech and Text Embedding Spaces
• Word Segmentation on Discovered Phone Units with Dynamic Programming and Self-Supervised Scoring
• Learning Hierarchical Discrete Linguistic Units from Visually-Grounded Speech
• Multilingual and Unsupervised Subword Modeling for Zero-Resource Languages
• A Correspondence Variational Autoencoder for Unsupervised Acoustic Word Embeddings
• Exploiting Cross-Lingual Speaker and Phonetic Diversity for Unsupervised Subword Modeling
• Unsupervised neural and Bayesian models for zero-resource speech processing
• An Unsupervised Probability Model for Speech-to-Translation Alignment of Low-Resource Languages
• Semantic speech retrieval with a visually grounded model of untranscribed speech
• Multilingual Jointly Trained Acoustic and Written Word Embeddings
• Automatically Identifying Language Family from Acoustic Examples in Low Resource Scenarios
• Unsupervised Multimodal Word Discovery based on Double Articulation Analysis with Co-occurrence cues
• Tabula nearly rasa: Probing the Linguistic Knowledge of Character-Level Neural Language Models Trained on Unsegmented Text
• Exploiting Cross-Lingual Knowledge in Unsupervised Acoustic Modeling for Low-Resource Languages