End-to-End Speaker Diarization for an Unknown Number of Speakers with Encoder-Decoder Based Attractors

Abstract

End-to-end speaker diarization for an unknown number of speakers is addressed in this paper. Recently proposed end-to-end speaker diarization outperformed conventional clustering-based speaker diarization, but it has one drawback: it is less flexible in terms of the number of speakers. This paper proposes a method for encoder-decoder based attractor calculation (EDA), which first generates a flexible number of attractors from a speech embedding sequence. Then, the generated multiple attractors are multiplied by the speech embedding sequence to produce the same number of speaker activities. The speech embedding sequence is extracted using the conventional self-attentive end-to-end neural speaker diarization (SA-EEND) network. In a two-speaker condition, our method achieved a 2.69 % diarization error rate (DER) on simulated mixtures and a 8.07 % DER on the two-speaker subset of CALLHOME, while vanilla SA-EEND attained 4.56 % and 9.54 %, respectively. In unknown numbers of speakers conditions, our method attained a 15.29 % DER on CALLHOME, while the x-vector-based clustering method achieved a 19.43 % DER.

arXiv commentAccepted to INTERSPEECH 2020

References in corpus (5)

• Sequence to Sequence Learning with Neural Networks
• Deep attractor network for single-microphone speaker separation
• Speaker-independent Speech Separation with Deep Attractor Network
• End-to-End Neural Diarization: Reformulating Speaker Diarization as Simple Multi-label Classification
• Discriminative Neural Clustering for Speaker Diarisation

Cited by in corpus (4)

• WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
• Integrating end-to-end neural and clustering-based diarization: Getting the best of both worlds
• Separation Guided Speaker Diarization in Realistic Mismatched Conditions
• Advances in integration of end-to-end neural and clustering-based diarization for real conversational speech