Target-Speaker Voice Activity Detection: a Novel Approach for Multi-Speaker Diarization in a Dinner Party Scenario
arXiv:2005.07272 · doi:10.21437/Interspeech.2020-1602
Abstract
Speaker diarization for real-life scenarios is an extremely challenging problem. Widely used clustering-based diarization approaches perform rather poorly in such conditions, mainly due to the limited ability to handle overlapping speech. We propose a novel Target-Speaker Voice Activity Detection (TS-VAD) approach, which directly predicts an activity of each speaker on each time frame. TS-VAD model takes conventional speech features (e.g., MFCC) along with i-vectors for each speaker as inputs. A set of binary classification output layers produces activities of each speaker. I-vectors can be estimated iteratively, starting with a strong clustering-based diarization. We also extend the TS-VAD approach to the multi-microphone case using a simple attention mechanism on top of hidden representations extracted from the single-channel TS-VAD model. Moreover, post-processing strategies for the predicted speaker activity probabilities are investigated. Experiments on the CHiME-6 unsegmented data show that TS-VAD achieves state-of-the-art results outperforming the baseline x-vector-based system by more than 30% Diarization Error Rate (DER) abs.
arXiv commentAccepted to Interspeech 2020
References in corpus (3)
Cited by in corpus (28)
- Neural Target Speech Extraction: An Overview
- Encoder-Decoder Based Attractors for End-to-End Neural Diarization
- AVA-AVD: Audio-Visual Speaker Diarization in the Wild
- TS-SEP: Joint Diarization and Separation Conditioned on Estimated Speaker Embeddings
- Online Neural Diarization of Unlimited Numbers of Speakers Using Global and Local Attractors
- The VoxCeleb Speaker Recognition Challenge: A Retrospective
- USTC-NELSLIP System Description for DIHARD-III Challenge
- The DKU-DukeECE-Lenovo System for the Diarization Task of the 2021 VoxCeleb Speaker Recognition Challenge
- Multi-target Extractor and Detector for Unknown-number Speaker Diarization
- Microphone Array Signal Processing and Deep Learning for Speech Enhancement
- TSUP Speaker Diarization System for Conversational Short-phrase Speaker Diarization Challenge
- Separation Guided Speaker Diarization in Realistic Mismatched Conditions
- Sequence to Multi-Sequence Learning via Conditional Chain Mapping for Mixture Signals
- Speaker activity driven neural speech extraction
- Investigation of End-To-End Speaker-Attributed ASR for Continuous Multi-Talker Recordings
- A Teacher-Student approach for extracting informative speaker embeddings from speech mixtures
- Towards Measuring and Scoring Speaker Diarization Fairness
- Target Speaker Verification with Selective Auditory Attention for Single and Multi-talker Speech
- Target-speaker Voice Activity Detection with Improved I-Vector Estimation for Unknown Number of Speaker
- Towards Neural Diarization for Unlimited Numbers of Speakers Using Global and Local Attractors
- The DKU-Duke-Lenovo System Description for the Third DIHARD Speech Diarization Challenge
- A Comparative Study of Modular and Joint Approaches for Speaker-Attributed ASR on Monaural Long-Form Audio
- Block-Online Guided Source Separation
- End-to-End Speaker Diarization as Post-Processing
- Speaker Embedding-aware Neural Diarization for Flexible Number of Speakers with Textual Information
- End-to-End Speaker Diarization Conditioned on Speech Activity and Overlap Detection
- Enrollment-less training for personalized voice activity detection
- Semi-Supervised Training with Pseudo-Labeling for End-to-End Neural Diarization