Online Neural Diarization of Unlimited Numbers of Speakers Using Global and Local Attractors

Abstract

A method to perform offline and online speaker diarization for an unlimited number of speakers is described in this paper. End-to-end neural diarization (EEND) has achieved overlap-aware speaker diarization by formulating it as a multi-label classification problem. It has also been extended for a flexible number of speakers by introducing speaker-wise attractors. However, the output number of speakers of attractor-based EEND is empirically capped; it cannot deal with cases where the number of speakers appearing during inference is higher than that during training because its speaker counting is trained in a fully supervised manner. Our method, EEND-GLA, solves this problem by introducing unsupervised clustering into attractor-based EEND. In the method, the input audio is first divided into short blocks, then attractor-based diarization is performed for each block, and finally, the results of each block are clustered on the basis of the similarity between locally-calculated attractors. While the number of output speakers is limited within each block, the total number of speakers estimated for the entire input can be higher than the limitation. To use EEND-GLA in an online manner, our method also extends the speaker-tracing buffer, which was originally proposed to enable online inference of conventional EEND. We introduce a block-wise buffer update to make the speaker-tracing buffer compatible with EEND-GLA. Finally, to improve online diarization, our method improves the buffer update method and revisits the variable chunk-size training of EEND. The experimental results demonstrate that EEND-GLA can perform speaker diarization of an unseen number of speakers in both offline and online inferences.

Accepted to IEEE/ACM TASLP

References in corpus (11)

• Semantic Instance Segmentation via Deep Metric Learning
• Target-Speaker Voice Activity Detection: a Novel Approach for Multi-Speaker Diarization in a Dinner Party Scenario
• ECAPA-TDNN Embeddings for Speaker Diarization
• Auto-Tuning Spectral Clustering for Speaker Diarization Using Normalized Maximum Eigengap
• Overlap-aware diarization: resegmentation using neural end-to-end overlapped speech detection
• Encoder-Decoder Based Attractors for End-to-End Neural Diarization
• Neural Speaker Diarization with Speaker-Wise Chain Rule
• End-to-end Neural Diarization: From Transformer to Conformer
• BW-EDA-EEND: Streaming End-to-End Neural Speaker Diarization for a Variable Number of Speakers
• Audiovisual speaker diarization of TV series
• Multi-target Extractor and Detector for Unknown-number Speaker Diarization

Cited by in corpus (1)

• End-to-End Integration of Speech Separation and Voice Activity Detection for Low-Latency Diarization of Telephone Conversations