Neural Speaker Diarization with Speaker-Wise Chain Rule

Abstract

Speaker diarization is an essential step for processing multi-speaker audio. Although an end-to-end neural diarization (EEND) method achieved state-of-the-art performance, it is limited to a fixed number of speakers. In this paper, we solve this fixed number of speaker issue by a novel speaker-wise conditional inference method based on the probabilistic chain rule. In the proposed method, each speaker's speech activity is regarded as a single random variable, and is estimated sequentially conditioned on previously estimated other speakers' speech activities. Similar to other sequence-to-sequence models, the proposed method produces a variable number of speakers with a stop sequence condition. We evaluated the proposed method on multi-speaker audio recordings of a variable number of speakers. Experimental results show that the proposed method can correctly produce diarization results with a variable number of speakers and outperforms the state-of-the-art end-to-end speaker diarization methods in terms of diarization error rate.

Submitted to Interspeech 2020

References in corpus (3)

• Sequence to Sequence Learning with Neural Networks
• Speaker-independent Speech Separation with Deep Attractor Network
• Discriminative Neural Clustering for Speaker Diarisation

Cited by in corpus (17)

• WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
• Encoder-Decoder Based Attractors for End-to-End Neural Diarization
• BW-EDA-EEND: Streaming End-to-End Neural Speaker Diarization for a Variable Number of Speakers
• The Hitachi-JHU DIHARD III System: Competitive End-to-End Neural Diarization and X-Vector Clustering Systems Combined by DOVER-Lap
• Online Neural Diarization of Unlimited Numbers of Speakers Using Global and Local Attractors
• Speaker Recognition Based on Deep Learning: An Overview
• Online End-to-End Neural Diarization with Speaker-Tracing Buffer
• TSUP Speaker Diarization System for Conversational Short-phrase Speaker Diarization Challenge
• Acoustic Event Detection with Classifier Chains
• Separation Guided Speaker Diarization in Realistic Mismatched Conditions
• Sequence to Multi-Sequence Learning via Conditional Chain Mapping for Mixture Signals
• Multi-Speaker ASR Combining Non-Autoregressive Conformer CTC and Conditional Speaker Chain
• Online Streaming End-to-End Neural Diarization Handling Overlapping Speech and Flexible Numbers of Speakers
• Towards Neural Diarization for Unlimited Numbers of Speakers Using Global and Local Attractors
• End-to-End Speaker Diarization as Post-Processing
• End-to-End Speaker Diarization Conditioned on Speech Activity and Overlap Detection
• Semi-Supervised Training with Pseudo-Labeling for End-to-End Neural Diarization