Attention-based Interactive Disentangling Network for Instance-level Emotional Voice Conversion

Abstract

Emotional Voice Conversion aims to manipulate a speech according to a given emotion while preserving non-emotion components. Existing approaches cannot well express fine-grained emotional attributes. In this paper, we propose an Attention-based Interactive diseNtangling Network (AINN) that leverages instance-wise emotional knowledge for voice conversion. We introduce a two-stage pipeline to effectively train our network: Stage I utilizes inter-speech contrastive learning to model fine-grained emotion and intra-speech disentanglement learning to better separate emotion and content. In Stage II, we propose to regularize the conversion with a multi-view consistency mechanism. This technique helps us transfer fine-grained emotion and maintain speech content. Extensive experiments show that our AINN outperforms state-of-the-arts in both objective and subjective metrics.

arXiv commentAccepted by INTERSPEECH 2023

References in corpus (7)

• HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis
• AUTOVC: Zero-Shot Voice Style Transfer with Only Autoencoder Loss
• Self-supervised Image-specific Prototype Exploration for Weakly Supervised Semantic Segmentation
• Emotion Intensity and its Control for Emotional Voice Conversion
• Nonparallel Emotional Speech Conversion
• VAW-GAN for Disentanglement and Recomposition of Emotional Elements in Speech
• Cross-speaker emotion disentangling and transfer for end-to-end speech synthesis