In defence of metric learning for speaker recognition

Abstract

The objective of this paper is 'open-set' speaker recognition of unseen speakers, where ideal embeddings should be able to condense information into a compact utterance-level representation that has small intra-speaker and large inter-speaker distance. A popular belief in speaker recognition is that networks trained with classification objectives outperform metric learning methods. In this paper, we present an extensive evaluation of most popular loss functions for speaker recognition on the VoxCeleb dataset. We demonstrate that the vanilla triplet loss shows competitive performance compared to classification-based losses, and those trained with our proposed metric learning objective outperform state-of-the-art methods.

arXiv commentThe code can be found at https://github.com/clovaai/voxceleb_trainer

References in corpus (14)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Prototypical Networks for Few-shot Learning
• Instance Normalization: The Missing Ingredient for Fast Stylization
• VoxCeleb2: Deep Speaker Recognition
• VoxCeleb: a large-scale speaker identification dataset
• Attentive Statistics Pooling for Deep Speaker Embedding
• Unified Hypersphere Embedding for Speaker Recognition
• BUT System Description to VoxCeleb Speaker Recognition Challenge 2019
• NSML: A Machine Learning Platform That Enables You to Focus on Your Models
• Few Shot Speaker Recognition using Deep Neural Networks
• Margin Matters: Towards More Discriminative Deep Neural Network Embeddings for Speaker Recognition
• Channel adversarial training for speaker verification and diarization
• Pairwise Discriminative Neural PLDA for Speaker Verification
• DropClass and DropAdapt: Dropping classes for deep speaker representation learning

Cited by in corpus (74)

• Spot the conversation: speaker diarisation in the wild
• Is Someone Speaking? Exploring Long-term Temporal Features for Audio-visual Active Speaker Detection
• Clova Baseline System for the VoxCeleb Speaker Recognition Challenge 2020
• VoxSRC 2020: The Second VoxCeleb Speaker Recognition Challenge
• Bias in Automated Speaker Recognition
• Self-Supervised Learning Based Domain Adaptation for Robust Speaker Verification
• Seeing voices and hearing voices: learning discriminative embeddings using cross-modal self-supervision
• Attack on practical speaker verification system using universal adversarial perturbations
• AVA-AVD: Audio-Visual Speaker Diarization in the Wild
• Semi-Supervised Contrastive Learning with Generalized Contrastive Loss and Its Application to Speaker Recognition
• Leveraging ASR Pretrained Conformers for Speaker Verification through Transfer Learning and Knowledge Distillation
• C3-DINO: Joint Contrastive and Non-contrastive Self-Supervised Learning for Speaker Verification
• Attention Back-end for Automatic Speaker Verification with Multiple Enrollment Utterances
• Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation
• Families In Wild Multimedia: A Multimodal Database for Recognizing Kinship
• The VoxCeleb Speaker Recognition Challenge: A Retrospective
• EfficientTDNN: Efficient Architecture Search for Speaker Recognition
• NeXt-TDNN: Modernizing Multi-Scale Temporal Convolution Backbone for Speaker Verification
• Generalizing AUC Optimization to Multiclass Classification for Audio Segmentation With Limited Training Data
• Cross-Lingual Speaker Verification with Domain-Balanced Hard Prototype Mining and Language-Dependent Score Normalization
• Why do Angular Margin Losses work well for Semi-Supervised Anomalous Sound Detection?
• Symmetric Saliency-based Adversarial Attack To Speaker Identification
• Improving fairness in speaker verification via Group-adapted Fusion Network
• A Quantum Kernel Learning Approach to Acoustic Modeling for Spoken Command Recognition
• openFEAT: Improving Speaker Identification by Open-set Few-shot Embedding Adaptation with Transformer
• End-to-End Zero-Shot Voice Conversion with Location-Variable Convolutions
• Contrastive-mixup learning for improved speaker verification
• Speaker Recognition Based on Deep Learning: An Overview
• Zero-shot text-to-speech synthesis conditioned using self-supervised speech representation model
• Label-Efficient Self-Supervised Speaker Verification With Information Maximization and Contrastive Learning
• Towards Lightweight Speaker Verification via Adaptive Neural Network Quantization
• Cluster-Guided Unsupervised Domain Adaptation for Deep Speaker Embedding
• SLMIA-SR: Speaker-Level Membership Inference Attacks against Speaker Recognition Systems
• Sparsely Overlapped Speech Training in the Time Domain: Joint Learning of Target Speech Separation and Personal VAD Benefits
• Adversarial Speaker Distillation for Countermeasure Model on Automatic Speaker Verification
• CITISEN: A Deep Learning-Based Speech Signal-Processing Mobile Application
• The 2020 Personalized Voice Trigger Challenge: Open Database, Evaluation Metrics and the Baseline Systems
• Attention-based multi-channel speaker verification with ad-hoc microphone arrays
• LMD: A Learnable Mask Network to Detect Adversarial Examples for Speaker Verification
• Deep Neural Networks for Automatic Speaker Recognition Do Not Learn Supra-Segmental Temporal Features
• Bootstrap Equilibrium and Probabilistic Speaker Representation Learning for Self-supervised Speaker Verification
• Deep Bayes Factor Scoring for Authorship Verification
• Self-supervised Speaker Recognition Training Using Human-Machine Dialogues
• Improving Speaker Identification for Shared Devices by Adapting Embeddings to Speaker Subsets
• Adversarial Reweighting for Speaker Verification Fairness
• Experimenting with Additive Margins for Contrastive Self-Supervised Speaker Verification
• HiSSNet: Sound Event Detection and Speaker Identification via Hierarchical Prototypical Networks for Low-Resource Headphones
• Residual Information in Deep Speaker Embedding Architectures
• Voting for the right answer: Adversarial defense for speaker verification
• An iterative framework for self-supervised deep speaker representation learning
• Intra-class variation reduction of speaker representation in disentanglement framework
• Masked Proxy Loss For Text-Independent Speaker Verification
• SpeechNAS: Towards Better Trade-off between Latency and Accuracy for Large-Scale Speaker Verification
• Speech Rhythm-Based Speaker Embeddings Extraction from Phonemes and Phoneme Duration for Multi-Speaker Speech Synthesis
• Self-supervised Text-independent Speaker Verification using Prototypical Momentum Contrastive Learning
• Automatic Face Understanding: Recognizing Families in Photos
• Temporal Dynamic Convolutional Neural Network for Text-Independent Speaker Verification and Phonemetic Analysis
• SVEva Fair: A Framework for Evaluating Fairness in Speaker Verification
• Detection and Evaluation of human and machine generated speech in spoofing attacks on automatic speaker verification systems
• MIRNet: Learning multiple identities representations in overlapped speech
• Unsupervised Representation Learning for Speaker Recognition via Contrastive Equilibrium Learning
• Momentum Contrast Speaker Representation Learning
• Improving Text-Independent Speaker Verification with Auxiliary Speakers Using Graph
• Query Expansion System for the VoxCeleb Speaker Recognition Challenge 2020
• Inclusive Speaker Verification with Adaptive thresholding
• Supervised attention for speaker recognition
• Coupling a generative model with a discriminative learning framework for speaker verification
• Non-local convolutional neural networks (nlcnn) for speaker recognition
• Tongji University Undergraduate Team for the VoxCeleb Speaker Recognition Challenge2020
• Tongji University Team for the VoxCeleb Speaker Recognition Challenge 2020
• Learning Metrics from Mean Teacher: A Supervised Learning Method for Improving the Generalization of Speaker Verification System
• Designing Neural Speaker Embeddings with Meta Learning
• Self-Supervised Frameworks for Speaker Verification via Bootstrapped Positive Sampling
• Duality Temporal-channel-frequency Attention Enhanced Speaker Representation Learning