Axial Residual Networks for CycleGAN-based Voice Conversion
arXiv:2102.08075
Abstract
We propose a novel architecture and improved training objectives for non-parallel voice conversion. Our proposed CycleGAN-based model performs a shape-preserving transformation directly on a high frequency-resolution magnitude spectrogram, converting its style (i.e. speaker identity) while preserving the speech content. Throughout the entire conversion process, the model does not resort to compressed intermediate representations of any sort (e.g. mel spectrogram, low resolution spectrogram, decomposed network feature). We propose an efficient axial residual block architecture to support this expensive procedure and various modifications to the CycleGAN losses to stabilize the training process. We demonstrate via experiments that our proposed model outperforms Scyclone and shows a comparable or better performance to that of CycleGAN-VC2 even without employing a neural vocoder.
References in corpus (10)
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Spectral Normalization for Generative Adversarial Networks
- MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis
- Axial Attention in Multidimensional Transformers
- Pay Less Attention with Lightweight and Dynamic Convolutions
- Parallel-Data-Free Voice Conversion Using Cycle-Consistent Adversarial Networks
- ACVAE-VC: Non-parallel many-to-many voice conversion with auxiliary classifier variational autoencoder
- Voice Transformer Network: Sequence-to-Sequence Voice Conversion Using Transformer with Text-to-Speech Pretraining
- Parallel Tacotron: Non-Autoregressive and Controllable TTS
- Scyclone: High-Quality and Parallel-Data-Free Voice Conversion Using Spectrogram and Cycle-Consistent Adversarial Networks