Double Articulation Analyzer with Prosody for Unsupervised Word and Phoneme Discovery

Abstract

Infants acquire words and phonemes from unsegmented speech signals using segmentation cues, such as distributional, prosodic, and co-occurrence cues. Many pre-existing computational models that represent the process tend to focus on distributional or prosodic cues. This paper proposes a nonparametric Bayesian probabilistic generative model called the prosodic hierarchical Dirichlet process-hidden language model (Prosodic HDP-HLM). Prosodic HDP-HLM, an extension of HDP-HLM, considers both prosodic and distributional cues within a single integrative generative model. We conducted three experiments on different types of datasets, and demonstrate the validity of the proposed method. The results show that the Prosodic DAA successfully uses prosodic cues and outperforms a method that solely uses distributional cues. The main contributions of this study are as follows: 1) We develop a probabilistic generative model for time series data including prosody that potentially has a double articulation structure; 2) We propose the Prosodic DAA by deriving the inference procedure for Prosodic HDP-HLM and show that Prosodic DAA can discover words directly from continuous human speech signals using statistical information and prosodic information in an unsupervised manner; 3) We show that prosodic cues contribute to word segmentation more in naturally distributed case words, i.e., they follow Zipf's law.

arXiv comment11 pages, Submitted to IEEE Transactions on Cognitive and Developmental Systems

References in corpus (10)

• Auto-Encoding Variational Bayes
• Representations of language in a model of visually grounded speech signal
• A segmental framework for fully-unsupervised large-vocabulary speech recognition
• Spatial Concept Acquisition for a Mobile Robot that Integrates Self-Localization and Unsupervised Word Discovery from Spoken Sentences
• Improved and Scalable Online Learning of Spatial Concepts and Language Models with Mapping
• Nonparametric Bayesian Double Articulation Analyzer for Direct Language Acquisition from Continuous Speech Signals
• A Whole Brain Probabilistic Generative Model: Toward Realizing Cognitive Architectures for Developmental Robots
• Perception of prosodic variation for speech synthesis using an unsupervised discrete representation of F0
• A Hierarchical Subspace Model for Language-Attuned Acoustic Unit Discovery
• Towards localisation of keywords in speech using weak supervision

Cited by in corpus (2)

• Word Segmentation on Discovered Phone Units with Dynamic Programming and Self-Supervised Scoring
• On Parallelism in Music and Language: A Perspective from Symbol Emergence Systems based on Probabilistic Generative Models