Improving Unsupervised Subword Modeling via Disentangled Speech Representation Learning and Transformation

Abstract

This study tackles unsupervised subword modeling in the zero-resource scenario, learning frame-level speech representation that is phonetically discriminative and speaker-invariant, using only untranscribed speech for target languages. Frame label acquisition is an essential step in solving this problem. High quality frame labels should be in good consistency with golden transcriptions and robust to speaker variation. We propose to improve frame label acquisition in our previously adopted deep neural network-bottleneck feature (DNN-BNF) architecture by applying the factorized hierarchical variational autoencoder (FHVAE). FHVAEs learn to disentangle linguistic content and speaker identity information encoded in speech. By discarding or unifying speaker information, speaker-invariant features are learned and fed as inputs to DPGMM frame clustering and DNN-BNF training. Experiments conducted on ZeroSpeech 2017 show that our proposed approaches achieve $2.4\%$ and $0.6\%$ absolute ABX error rate reductions in across- and within-speaker conditions, comparing to the baseline DNN-BNF system without applying FHVAEs. Our proposed approaches significantly outperform vocal tract length normalization in improving frame labeling and subword modeling.

5 pages, 3 figures, accepted for publication in INTERSPEECH 2019, Graz, Austria

References in corpus (1)

• Unsupervised Learning of Disentangled and Interpretable Representations from Sequential Data

Cited by in corpus (4)

• Combining Adversarial Training and Disentangled Speech Representation for Robust Zero-Resource Subword Modeling
• Unsupervised Subword Modeling Using Autoregressive Pretraining and Cross-Lingual Phone-Aware Modeling
• The effectiveness of unsupervised subword modeling with autoregressive and cross-lingual phone-aware networks
• Deep Generative Variational Autoencoding for Replay Spoof Detection in Automatic Speaker Verification