Fine-tuning wav2vec2 for speaker recognition
arXiv:2109.15053 · doi:10.1109/ICASSP43922.2022.9746952
Abstract
This paper explores applying the wav2vec2 framework to speaker recognition instead of speech recognition. We study the effectiveness of the pre-trained weights on the speaker recognition task, and how to pool the wav2vec2 output sequence into a fixed-length speaker embedding. To adapt the framework to speaker recognition, we propose a single-utterance classification variant with CE or AAM softmax loss, and an utterance-pair classification variant with BCE loss. Our best performing variant, w2v2-aam, achieves a 1.88% EER on the extended voxceleb1 test set compared to 1.69% EER with an ECAPA-TDNN baseline. Code is available at https://github.com/nikvaessen/w2v2-speaker.
arXiv commentaccepted to ICASSP 2022
References in corpus (9)
- Adam: A Method for Stochastic Optimization
- SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
- Gaussian Error Linear Units (GELUs)
- VoxCeleb2: Deep Speaker Recognition
- VoxCeleb: a large-scale speaker identification dataset
- ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification
- SpeechBrain: A General-Purpose Speech Toolkit
- Reducing Transformer Depth on Demand with Structured Dropout
- The Role of Phonetic Units in Speech Emotion Recognition
Cited by in corpus (9)
- Wav2vec-based Detection and Severity Level Classification of Dysarthria from Speech
- Leveraging ASR Pretrained Conformers for Speaker Verification through Transfer Learning and Knowledge Distillation
- Investigation of Self-supervised Pre-trained Models for Classification of Voice Quality from Speech and Neck Surface Accelerometer Signals
- Training speaker recognition systems with limited data
- Leveraging Semantic Information for Efficient Self-Supervised Emotion Recognition with Audio-Textual Distilled Models
- Advancing Audio Emotion and Intent Recognition with Large Pre-Trained Models and Bayesian Inference
- Residual Information in Deep Speaker Embedding Architectures
- Self-supervised Reflective Learning through Self-distillation and Online Clustering for Speaker Representation Learning
- XLSR-Kanformer: A KAN-Intergrated model for Synthetic Speech Detection