Fine-tuning wav2vec2 for speaker recognition

Abstract

This paper explores applying the wav2vec2 framework to speaker recognition instead of speech recognition. We study the effectiveness of the pre-trained weights on the speaker recognition task, and how to pool the wav2vec2 output sequence into a fixed-length speaker embedding. To adapt the framework to speaker recognition, we propose a single-utterance classification variant with CE or AAM softmax loss, and an utterance-pair classification variant with BCE loss. Our best performing variant, w2v2-aam, achieves a 1.88% EER on the extended voxceleb1 test set compared to 1.69% EER with an ECAPA-TDNN baseline. Code is available at https://github.com/nikvaessen/w2v2-speaker.

accepted to ICASSP 2022

References in corpus (3)

• SpeechBrain: A General-Purpose Speech Toolkit
• Reducing Transformer Depth on Demand with Structured Dropout
• The Role of Phonetic Units in Speech Emotion Recognition

Cited by in corpus (9)

• Wav2vec-based Detection and Severity Level Classification of Dysarthria from Speech
• Leveraging ASR Pretrained Conformers for Speaker Verification through Transfer Learning and Knowledge Distillation
• Investigation of Self-supervised Pre-trained Models for Classification of Voice Quality from Speech and Neck Surface Accelerometer Signals
• Training speaker recognition systems with limited data
• Leveraging Semantic Information for Efficient Self-Supervised Emotion Recognition with Audio-Textual Distilled Models
• Advancing Audio Emotion and Intent Recognition with Large Pre-Trained Models and Bayesian Inference
• Residual Information in Deep Speaker Embedding Architectures
• Self-supervised Reflective Learning through Self-distillation and Online Clustering for Speaker Representation Learning
• XLSR-Kanformer: A KAN-Intergrated model for Synthetic Speech Detection