Non-parallel Voice Conversion System with WaveNet Vocoder and Collapsed Speech Suppression

Abstract

In this paper, we integrate a simple non-parallel voice conversion (VC) system with a WaveNet (WN) vocoder and a proposed collapsed speech suppression technique. The effectiveness of WN as a vocoder for generating high-fidelity speech waveforms on the basis of acoustic features has been confirmed in recent works. However, when combining the WN vocoder with a VC system, the distorted acoustic features, acoustic and temporal mismatches, and exposure bias usually lead to significant speech quality degradation, making WN generate some very noisy speech segments called collapsed speech. To tackle the problem, we take conventional-vocoder-generated speech as the reference speech to derive a linear predictive coding distribution constraint (LPCDC) to avoid the collapsed speech problem. Furthermore, to mitigate the negative effects introduced by the LPCDC, we propose a collapsed speech segment detector (CSSD) to ensure that the LPCDC is only applied to the problematic segments to limit the loss of quality to short periods. Objective and subjective evaluations are conducted, and the experimental results confirm the effectiveness of the proposed method, which further improves the speech quality of our previous non-parallel VC system submitted to Voice Conversion Challenge 2018.

13 pages, 13 figures, 1 table, accepted to publish in IEEE Access

References in corpus (8)

• Adam: A Method for Stochastic Optimization
• WaveNet: A Generative Model for Raw Audio
• Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
• Sequence Level Training with Recurrent Neural Networks
• SampleRNN: An Unconditional End-to-End Neural Audio Generation Model
• Parallel WaveNet: Fast High-Fidelity Speech Synthesis
• Parallel-Data-Free Voice Conversion Using Cycle-Consistent Adversarial Networks
• Refined WaveNet Vocoder for Variational Autoencoder Based Voice Conversion

Cited by in corpus (3)

• Quasi-Periodic WaveNet: An Autoregressive Raw Waveform Generative Model with Pitch-dependent Dilated Convolution Neural Network
• Quasi-Periodic Parallel WaveGAN: A Non-autoregressive Raw Waveform Generative Model with Pitch-dependent Dilated Convolution Neural Network
• A Cyclical Post-filtering Approach to Mismatch Refinement of Neural Vocoder for Text-to-speech Systems