Probing the phonetic and phonological knowledge of tones in Mandarin TTS models
arXiv:1912.10915
Abstract
This study probes the phonetic and phonological knowledge of lexical tones in TTS models through two experiments. Controlled stimuli for testing tonal coarticulation and tone sandhi in Mandarin were fed into Tacotron 2 and WaveGlow to generate speech samples, which were subject to acoustic analysis and human evaluation. Results show that both baseline Tacotron 2 and Tacotron 2 with BERT embeddings capture the surface tonal coarticulation patterns well but fail to consistently apply the Tone-3 sandhi rule to novel sentences. Incorporating pre-trained BERT embeddings into Tacotron 2 improves the naturalness and prosody performance, and yields better generalization of Tone-3 sandhi rules to novel complex sentences, although the overall accuracy for Tone-3 sandhi was still low. Given that TTS models do capture some linguistic phenomena, it is argued that they can be used to generate and validate certain linguistic hypotheses. On the other hand, it is also suggested that linguistically informed stimuli should be included in the training and the evaluation of TTS models.
arXiv commentSubmitted to Speech Prosody 2020
References in corpus (7)
- FastSpeech: Fast, Robust and Controllable Text to Speech
- RNNs as psycholinguistic subjects: Syntactic state and grammatical dependency
- Dynamic Prosody Generation for Speech Synthesis using Linguistics-Driven Acoustic Embedding Selection
- Towards Transfer Learning for End-to-End Speech Synthesis from Deep Pre-Trained Language Models
- Neural Language Models as Psycholinguistic Subjects: Representations of Syntactic State
- A Variational Prosody Model for Mapping the Context-Sensitive Variation of Functional Prosodic Prototypes
- Representation Learning for Discovering Phonemic Tone Contours