Deep Speaker: an End-to-End Neural Speaker Embedding System

Abstract

We present Deep Speaker, a neural speaker embedding system that maps utterances to a hypersphere where speaker similarity is measured by cosine similarity. The embeddings generated by Deep Speaker can be used for many tasks, including speaker identification, verification, and clustering. We experiment with ResCNN and GRU architectures to extract the acoustic features, then mean pool to produce utterance-level speaker embeddings, and train using triplet loss based on cosine similarity. Experiments on three distinct datasets suggest that Deep Speaker outperforms a DNN-based i-vector baseline. For example, Deep Speaker reduces the verification equal error rate by 50% (relatively) and improves the identification accuracy by 60% (relatively) on a text-independent dataset. We also present results that suggest adapting from a model trained with Mandarin can improve accuracy for English speaker recognition.

References in corpus (2)

• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Deep Speech 2: End-to-End Speech Recognition in English and Mandarin

Cited by in corpus (111)

• VoxCeleb2: Deep Speaker Recognition
• Attentive Statistics Pooling for Deep Speaker Embedding
• Deep Voice 2: Multi-Speaker Neural Text-to-Speech
• Neural Voice Cloning with a Few Samples
• AdaSpeech: Adaptive Text to Speech for Custom Voice
• Deep Representation Learning in Speech Processing: Challenges, Recent Advances, and Future Trends
• Adversarial Speaker Verification
• Neural Predictive Coding using Convolutional Neural Networks towards Unsupervised Learning of Speaker Characteristics
• Learning Efficient Representations for Keyword Spotting with Triplet Loss
• Temporal Modeling Approaches for Large-scale Youtube-8M Video Understanding
• Improving Multi-Scale Aggregation Using Feature Pyramid Module for Robust Speaker Verification of Variable-Duration Utterances
• Exploring the Encoding Layer and Loss Function in End-to-End Speaker and Language Recognition System
• Spatial Pyramid Encoding with Convex Length Normalization for Text-Independent Speaker Verification
• AM-MobileNet1D: A Portable Model for Speaker Recognition
• Modeling Multi-speaker Latent Space to Improve Neural TTS: Quick Enrolling New Speaker and Enhancing Premium Voice
• Generalized End-to-End Loss for Speaker Verification
• Multi-source Domain Adaptation for Text-independent Forensic Speaker Recognition
• Multi-task Learning for Speaker Verification and Voice Trigger Detection
• iQIYI-VID: A Large Dataset for Multi-modal Person Identification
• Attention Back-end for Automatic Speaker Verification with Multiple Enrollment Utterances
• Concept Learning through Deep Reinforcement Learning with Memory-Augmented Neural Networks
• AdaDurIAN: Few-shot Adaptation for Neural Text-to-Speech with DurIAN
• Defend Data Poisoning Attacks on Voice Authentication
• Few-Shot Speaker Identification Using Lightweight Prototypical Network with Feature Grouping and Interaction
• Additive Margin SincNet for Speaker Recognition
• Utterance-level Aggregation For Speaker Recognition In The Wild
• Large Margin Softmax Loss for Speaker Verification
• Insights into End-to-End Learning Scheme for Language Identification
• Meta-Learning for Short Utterance Speaker Recognition with Imbalance Length Pairs
• AutoSpeech: Neural Architecture Search for Speaker Recognition
• ResNeXt and Res2Net Structures for Speaker Verification
• Machine Speech Chain with One-shot Speaker Adaptation
• Improving fairness in speaker verification via Group-adapted Fusion Network
• Speaker attribution with voice profiles by graph-based semi-supervised learning
• Voice-Face Cross-modal Matching and Retrieval: A Benchmark
• Centroid-based deep metric learning for speaker recognition
• Speaker Recognition Based on Deep Learning: An Overview
• Improved Large-margin Softmax Loss for Speaker Diarisation
• Deep Normalization for Speaker Vectors
• Personal VAD: Speaker-Conditioned Voice Activity Detection
• Speaker Embedding Extraction with Phonetic Information
• Version Control of Speaker Recognition Systems
• SEC4SR: A Security Analysis Platform for Speaker Recognition
• Improving Speaker Identification for Shared Devices by Adapting Embeddings to Speaker Subsets
• From Speaker Verification to Multispeaker Speech Synthesis, Deep Transfer with Feedback Constraint
• Building a mixed-lingual neural TTS system with only monolingual data
• Analysis of Length Normalization in End-to-End Speaker Verification System
• Short utterance compensation in speaker verification via cosine-based teacher-student learning of speaker embeddings
• Triplet Based Embedding Distance and Similarity Learning for Text-independent Speaker Verification
• Channel adversarial training for cross-channel text-independent speaker recognition
• Intra-class variation reduction of speaker representation in disentanglement framework
• Label-efficient audio classification through multitask learning and self-supervision
• Deep Segment Attentive Embedding for Duration Robust Speaker Verification
• A Unified Deep Speaker Embedding Framework for Mixed-Bandwidth Speech Data
• Towards transformation-resilient provenance detection of digital media
• Deep Speaker Embedding Learning with Multi-Level Pooling for Text-Independent Speaker Verification
• Active Voice Authentication
• asya: Mindful verbal communication using deep learning
• MTGAN: Speaker Verification through Multitasking Triplet Generative Adversarial Networks
• An end-to-end approach for the verification problem: learning the right distance
• Training Multi-Task Adversarial Network for Extracting Noise-Robust Speaker Embedding
• A Study on Angular Based Embedding Learning for Text-independent Speaker Verification
• Structural sparsification for Far-field Speaker Recognition with GNA
• Speaker Diarization with Region Proposal Network
• Deep Factorization for Speech Signal
• STYLER: Style Factor Modeling with Rapidity and Robustness via Speech Decomposition for Expressive and Controllable Neural Text to Speech
• Confidence Calibration with Bounded Error Using Transformations
• Triplet loss based embeddings for forensic speaker identification in Spanish
• Deep Discriminative Feature Learning for Accent Recognition
• A study on the role of subsidiary information in replay attack spoofing detection
• SVEva Fair: A Framework for Evaluating Fairness in Speaker Verification
• Synth2Aug: Cross-domain speaker recognition with TTS synthesized speech
• Augmenting Images for ASR and TTS through Single-loop and Dual-loop Multimodal Chain Framework
• Incremental Machine Speech Chain Towards Enabling Listening while Speaking in Real-time
• Fusion of Embeddings Networks for Robust Combination of Text Dependent and Independent Speaker Recognition
• Adaptive Margin Circle Loss for Speaker Verification
• Adversarial Training of Denoising Diffusion Model Using Dual Discriminators for High-Fidelity Multi-Speaker TTS
• A Comparative Re-Assessment of Feature Extractors for Deep Speaker Embeddings
• End-to-end Triplet Loss based Emotion Embedding System for Speech Emotion Recognition
• MIRNet: Learning multiple identities representations in overlapped speech
• Evolutionary Algorithm Enhanced Neural Architecture Search for Text-Independent Speaker Verification
• End-to-End Trainable Self-Attentive Shallow Network for Text-Independent Speaker Verification
• Tackling real noisy reverberant meetings with all-neural source separation, counting, and diarization system
• Speaker Identification using EEG
• Comparison of Multiple Features and Modeling Methods for Text-dependent Speaker Verification
• Open-set Short Utterance Forensic Speaker Verification using Teacher-Student Network with Explicit Inductive Bias
• Partial AUC optimization based deep speaker embeddings with class-center learning for text-independent speaker verification
• Generative x-vectors for text-independent speaker verification
• Detection and Analysis of Content Creator Collaborations in YouTube Videos using Face- and Speaker-Recognition
• Text-Independent Speaker Verification Based on Deep Neural Networks and Segmental Dynamic Time Warping
• A Novel Learnable Dictionary Encoding Layer for End-to-End Language Identification
• End-to-end losses based on speaker basis vectors and all-speaker hard negative mining for speaker verification
• End-to-end Language Identification using NetFV and NetVLAD
• Exploring a Unified Attention-Based Pooling Framework for Speaker Verification
• Multiobjective Optimization Training of PLDA for Speaker Verification
• Additive Phoneme-aware Margin Softmax Loss for Language Recognition
• Shareable Representations for Search Query Understanding
• Deep Speaker Verification: Do We Need End to End?
• Multi-Channel Speaker Verification for Single and Multi-talker Speech
• End-to-End Residual CNN with L-GM Loss Speaker Verification System
• Non-local convolutional neural networks (nlcnn) for speaker recognition
• Self-Attentive Multi-Layer Aggregation with Feature Recalibration and Normalization for End-to-End Speaker Verification System
• Towards Robust Speaker Verification with Target Speaker Enhancement
• An Exploration of ECAPA-TDNN and x-vector Speaker Representations in Zero-shot Multi-speaker TTS
• AdaSpeech 2: Adaptive Text to Speech with Untranscribed Data
• Spoofing Speaker Verification Systems with Deep Multi-speaker Text-to-speech Synthesis
• Cross-domain Adaptation with Discrepancy Minimization for Text-independent Forensic Speaker Verification
• Multi-Task Network for Noise-Robust Keyword Spotting and Speaker Verification using CTC-based Soft VAD and Global Query Attention
• Length- and Noise-aware Training Techniques for Short-utterance Speaker Recognition
• Frame Aggregation and Multi-Modal Fusion Framework for Video-Based Person Recognition
• X-Vectors with Multi-Scale Aggregation for Speaker Diarization