Discriminative Neural Clustering for Speaker Diarisation

Abstract

In this paper, we propose Discriminative Neural Clustering (DNC) that formulates data clustering with a maximum number of clusters as a supervised sequence-to-sequence learning problem. Compared to traditional unsupervised clustering algorithms, DNC learns clustering patterns from training data without requiring an explicit definition of a similarity measure. An implementation of DNC based on the Transformer architecture is shown to be effective on a speaker diarisation task using the challenging AMI dataset. Since AMI contains only 147 complete meetings as individual input sequences, data scarcity is a significant issue for training a Transformer model for DNC. Accordingly, this paper proposes three data augmentation schemes: sub-sequence randomisation, input vector randomisation, and Diaconis augmentation, which generates new data samples by rotating the entire input sequence of L2-normalised speaker embeddings. Experimental results on AMI show that DNC achieves a reduction in speaker error rate (SER) of 29.4% relative to spectral clustering.

arXiv commentAccepted as a conference paper at the 8th IEEE Spoken Language Technology Workshop (SLT 2021)

References in corpus (3)

• Neural Machine Translation by Jointly Learning to Align and Translate
• LSTM based Similarity Measurement with Spectral Clustering for Speaker Diarization
• Monotonic Multihead Attention

Cited by in corpus (10)

• Neural Speaker Diarization with Speaker-Wise Chain Rule
• BW-EDA-EEND: Streaming End-to-End Neural Speaker Diarization for a Variable Number of Speakers
• Self-supervised Representation Learning With Path Integral Clustering For Speaker Diarization
• End-to-End Speaker Diarization for an Unknown Number of Speakers with Encoder-Decoder Based Attractors
• Speaker Recognition Based on Deep Learning: An Overview
• Attention is All You Need in Speech Separation
• DIHARD II is Still Hard: Experimental Results and Discussions from the DKU-LENOVO Team
• The DKU-Duke-Lenovo System Description for the Third DIHARD Speech Diarization Challenge
• End-to-End Speaker Diarization Conditioned on Speech Activity and Overlap Detection
• Semi-Supervised Training with Pseudo-Labeling for End-to-End Neural Diarization