BW-EDA-EEND: Streaming End-to-End Neural Speaker Diarization for a Variable Number of Speakers

Abstract

We present a novel online end-to-end neural diarization system, BW-EDA-EEND, that processes data incrementally for a variable number of speakers. The system is based on the Encoder-Decoder-Attractor (EDA) architecture of Horiguchi et al., but utilizes the incremental Transformer encoder, attending only to its left contexts and using block-level recurrence in the hidden states to carry information from block to block, making the algorithm complexity linear in time. We propose two variants: For unlimited-latency BW-EDA-EEND, which processes inputs in linear time, we show only moderate degradation for up to two speakers using a context size of 10 seconds compared to offline EDA-EEND. With more than two speakers, the accuracy gap between online and offline grows, but the algorithm still outperforms a baseline offline clustering diarization system for one to four speakers with unlimited context size, and shows comparable accuracy with context size of 10 seconds. For limited-latency BW-EDA-EEND, which produces diarization outputs block-by-block as audio arrives, we show accuracy comparable to the offline clustering-based system.

References in corpus (4)

• Deep attractor network for single-microphone speaker separation
• Overlap-aware diarization: resegmentation using neural end-to-end overlapped speech detection
• Neural Speaker Diarization with Speaker-Wise Chain Rule
• Discriminative Neural Clustering for Speaker Diarisation

Cited by in corpus (5)

• Encoder-Decoder Based Attractors for End-to-End Neural Diarization
• Online Neural Diarization of Unlimited Numbers of Speakers Using Global and Local Attractors
• Multitask Detection of Speaker Changes, Overlapping Speech and Voice Activity Using wav2vec 2.0
• ASR-Aware End-to-end Neural Diarization
• Online Streaming End-to-End Neural Diarization Handling Overlapping Speech and Flexible Numbers of Speakers