End-to-End Speaker Diarization Conditioned on Speech Activity and Overlap Detection

Abstract

In this paper, we present a conditional multitask learning method for end-to-end neural speaker diarization (EEND). The EEND system has shown promising performance compared with traditional clustering-based methods, especially in the case of overlapping speech. In this paper, to further improve the performance of the EEND system, we propose a novel multitask learning framework that solves speaker diarization and a desired subtask while explicitly considering the task dependency. We optimize speaker diarization conditioned on speech activity and overlap detection that are subtasks of speaker diarization, based on the probabilistic chain rule. Experimental results show that our proposed method can leverage a subtask to effectively model speaker diarization, and outperforms conventional EEND systems in terms of diarization error rate.

arXiv commentAccepted for SLT 2021

References in corpus (6)

• Target-Speaker Voice Activity Detection: a Novel Approach for Multi-Speaker Diarization in a Dinner Party Scenario
• CHiME-6 Challenge:Tackling Multispeaker Speech Recognition for Unsegmented Recordings
• LSTM based Similarity Measurement with Spectral Clustering for Speaker Diarization
• Neural Speaker Diarization with Speaker-Wise Chain Rule
• Discriminative Neural Clustering for Speaker Diarisation
• Online End-to-End Neural Diarization with Speaker-Tracing Buffer