VoiceFilter: Targeted Voice Separation by Speaker-Conditioned Spectrogram Masking

Abstract

In this paper, we present a novel system that separates the voice of a target speaker from multi-speaker signals, by making use of a reference signal from the target speaker. We achieve this by training two separate neural networks: (1) A speaker recognition network that produces speaker-discriminative embeddings; (2) A spectrogram masking network that takes both noisy spectrogram and speaker embedding as input, and produces a mask. Our system significantly reduces the speech recognition WER on multi-speaker signals, with minimal WER degradation on single-speaker signals.

arXiv commentTo appear in Interspeech 2019

References in corpus (4)

• VoxCeleb2: Deep Speaker Recognition
• Deep attractor network for single-microphone speaker separation
• Deep Extractor Network for Target Speaker Recovery From Single Channel Speech Mixtures
• Fully Supervised Speaker Diarization

Cited by in corpus (20)

• Target-Speaker Voice Activity Detection: a Novel Approach for Multi-Speaker Diarization in a Dinner Party Scenario
• Multi-modal Multi-channel Target Speech Separation
• SpEx+: A Complete Time Domain Speaker Extraction Network
• MAAS: Multi-modal Assignation for Active Speaker Detection
• Self-Supervised Learning from Contrastive Mixtures for Personalized Speech Enhancement
• Fully Supervised Speaker Diarization
• Continuous Speech Separation Using Speaker Inventory for Long Multi-talker Recording
• Multi-stage Speaker Extraction with Utterance and Frame-Level Reference Signals
• Rethinking the Separation Layers in Speech Separation Networks
• Noise Tokens: Learning Neural Noise Templates for Environment-Aware Speech Enhancement
• Auditory Separation of a Conversation from Background via Attentional Gating
• Personalized Speech Enhancement through Self-Supervised Data Augmentation and Purification
• Multimodal Target Speech Separation with Voice and Face References
• Speakerfilter-Pro: an improved target speaker extractor combines the time domain and frequency domain
• Speaker Separation Using Speaker Inventories and Estimated Speech
• Self-supervised Text-independent Speaker Verification using Prototypical Momentum Contrastive Learning
• Gender domain adaptation for automatic speech recognition task
• Speaker Representation Learning using Global Context Guided Channel and Time-Frequency Transformations
• Far-Field Automatic Speech Recognition
• Active Speakers in Context