Zero-shot Singing Technique Conversion
arXiv:2111.08839
Abstract
In this paper we propose modifications to the neural network framework, AutoVC for the task of singing technique conversion. This includes utilising a pretrained singing technique encoder which extracts technique information, upon which a decoder is conditioned during training. By swapping out a source singer's technique information for that of the target's during conversion, the input spectrogram is reconstructed with the target's technique. We document the beneficial effects of omitting the latent loss, the importance of sequential training, and our process for fine-tuning the bottleneck. We also conducted a listening study where participants rate the specificity of technique-converted voices as well as their naturalness. From this we are able to conclude how effective the technique conversions are and how different conditions affect them, while assessing the model's ability to reconstruct its input data.
arXiv commentIn Proceedings of the 15th International Symposium on Computer Music Multidisciplinary Research (CMMR 2021), Tokyo, Japan, November 15-16, 2021
References in corpus (10)
- Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis
- Feed-Forward Networks with Attention Can Solve Some Long-Term Memory Problems
- AUTOVC: Zero-Shot Voice Style Transfer with Only Autoencoder Loss
- Wasserstein Auto-Encoders
- F0-consistent many-to-many non-parallel voice conversion via conditional autoencoder
- ACVAE-VC: Non-parallel many-to-many voice conversion with auxiliary classifier variational autoencoder
- Unsupervised Speech Decomposition via Triple Information Bottleneck
- An Exploratory Study on Perceptual Spaces of the Singing Voice
- Spectrogram Inpainting for Interactive Generation of Instrument Sounds
- Singing Voice Conversion with Disentangled Representations of Singer and Vocal Technique Using Variational Autoencoders