Online End-to-End Neural Diarization with Speaker-Tracing Buffer

Abstract

This paper proposes a novel online speaker diarization algorithm based on a fully supervised self-attention mechanism (SA-EEND). Online diarization inherently presents a speaker's permutation problem due to the possibility to assign speaker regions incorrectly across the recording. To circumvent this inconsistency, we proposed a speaker-tracing buffer mechanism that selects several input frames representing the speaker permutation information from previous chunks and stores them in a buffer. These buffered frames are stacked with the input frames in the current chunk and fed into a self-attention network. Our method ensures consistent diarization outputs across the buffer and the current chunk by checking the correlation between their corresponding outputs. Additionally, we trained SA-EEND with variable chunk-sizes to mitigate the mismatch between training and inference introduced by the speaker-tracing buffer mechanism. Experimental results, including online SA-EEND and variable chunk-size, achieved DERs of 12.54% for CALLHOME and 20.77% for CSJ with 1.4s actual latency.

arXiv commentAccepted to SLT 2021

References in corpus (2)

• End-to-End Neural Diarization: Reformulating Speaker Diarization as Simple Multi-label Classification
• Neural Speaker Diarization with Speaker-Wise Chain Rule

Cited by in corpus (4)

• Speaker Recognition Based on Deep Learning: An Overview
• Integrating end-to-end neural and clustering-based diarization: Getting the best of both worlds
• Online Streaming End-to-End Neural Diarization Handling Overlapping Speech and Flexible Numbers of Speakers
• End-to-End Speaker Diarization Conditioned on Speech Activity and Overlap Detection