VoxCeleb2: Deep Speaker Recognition

Abstract

The objective of this paper is speaker recognition under noisy and unconstrained conditions. We make two key contributions. First, we introduce a very large-scale audio-visual speaker recognition dataset collected from open-source media. Using a fully automated pipeline, we curate VoxCeleb2 which contains over a million utterances from over 6,000 speakers. This is several times larger than any publicly available speaker recognition dataset. Second, we develop and compare Convolutional Neural Network (CNN) models and training strategies that can effectively recognise identities from voice under various conditions. The models trained on the VoxCeleb2 dataset surpass the performance of previous works on a benchmark dataset by a significant margin.

arXiv commentTo appear in Interspeech 2018. The audio-visual dataset can be downloaded from http://www.robots.ox.ac.uk/~vgg/data/voxceleb2 . 1806.05622v2: minor fixes; 5 pages

References in corpus (7)

• In Defense of the Triplet Loss for Person Re-Identification
• VoxCeleb: a large-scale speaker identification dataset
• Return of the Devil in the Details: Delving Deep into Convolutional Nets
• MatConvNet - Convolutional Neural Networks for MATLAB
• Deep Speaker: an End-to-End Neural Speaker Embedding System
• MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition
• You said that?

Cited by in corpus (380)

• WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
• ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification
• The VIA Annotation Software for Images, Audio and Video
• SpeechBrain: A General-Purpose Speech Toolkit
• Multimodal Intelligence: Representation Learning, Information Fusion, and Applications
• Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis
• In defence of metric learning for speaker recognition
• MakeItTalk: Speaker-Aware Talking-Head Animation
• WILDS: A Benchmark of in-the-Wild Distribution Shifts
• ASVspoof 2019: spoofing countermeasures for the detection of synthesized, converted and replayed speech
• Spot the conversation: speaker diarisation in the wild
• Self-Supervised Learning for Videos: A Survey
• Is Someone Speaking? Exploring Long-term Temporal Features for Audio-visual Active Speaker Detection
• Auto-AVSR: Audio-Visual Speech Recognition with Automatic Labels
• MelNet: A Generative Model for Audio in the Frequency Domain
• Self-supervised learning of a facial attribute embedding from video
• Neural Head Reenactment with Latent Pose Descriptors
• The VoicePrivacy 2020 Challenge: Results and findings
• Fine-tuning wav2vec2 for speaker recognition
• Introducing the VoicePrivacy Initiative
• ECAPA-TDNN Embeddings for Speaker Diarization
• Clova Baseline System for the VoxCeleb Speaker Recognition Challenge 2020
• Any-to-Many Voice Conversion with Location-Relative Sequence-to-Sequence Modeling
• Emotional Speech-Driven Animation with Content-Emotion Disentanglement
• BUT System Description to VoxCeleb Speaker Recognition Challenge 2019
• FaceFilter: Audio-visual speech separation using still images
• HiCMAE: Hierarchical Contrastive Masked Autoencoder for Self-Supervised Audio-Visual Emotion Recognition
• Deep Representation Learning in Speech Processing: Challenges, Recent Advances, and Future Trends
• VoxSRC 2020: The Second VoxCeleb Speaker Recognition Challenge
• Integrating Frequency Translational Invariance in TDNNs and Frequency Positional Information in 2D ResNets to Enhance Speaker Verification
• Probing the Information Encoded in X-vectors
• Empowering Things with Intelligence: A Survey of the Progress, Challenges, and Opportunities in Artificial Intelligence of Things
• Self-supervised Learning of Interpretable Keypoints from Unlabelled Videos
• The IDLAB VoxSRC-20 Submission: Large Margin Fine-Tuning and Quality-Aware Score Calibration in DNN Based Speaker Verification
• Meta-TTS: Meta-Learning for Few-Shot Speaker Adaptive Text-to-Speech
• Imitating Arbitrary Talking Style for Realistic Audio-DrivenTalking Face Synthesis
• VoiceFilter: Targeted Voice Separation by Speaker-Conditioned Spectrogram Masking
• VoxSRC 2019: The first VoxCeleb Speaker Recognition Challenge
• Supervision by Registration and Triangulation for Landmark Detection
• Improving Multi-Scale Aggregation Using Feature Pyramid Module for Robust Speaker Verification of Variable-Duration Utterances
• Seeing voices and hearing voices: learning discriminative embeddings using cross-modal self-supervision
• Self-Supervised learning with cross-modal transformers for emotion recognition
• UniCon: Unified Context Network for Robust Active Speaker Detection
• VATLM: Visual-Audio-Text Pre-Training with Unified Masked Prediction for Speech Representation Learning
• The SpeakIn System for VoxCeleb Speaker Recognition Challange 2021
• LipSound2: Self-Supervised Pre-Training for Lip-to-Speech Reconstruction and Lip Reading
• Deep Person Generation: A Survey from the Perspective of Face, Pose and Cloth Synthesis
• Voice activity detection in the wild: A data-driven approach using teacher-student training
• Everybody's Talkin': Let Me Talk as You Want
• Spatial Pyramid Encoding with Convex Length Normalization for Text-Independent Speaker Verification
• Deep learning methods in speaker recognition: a review
• Few Shot Speaker Recognition using Deep Neural Networks
• The VOiCES from a Distance Challenge 2019 Evaluation Plan
• AVA-AVD: Audio-Visual Speaker Diarization in the Wild
• Deep multi-metric learning for text-independent speaker verification
• F3A-GAN: Facial Flow for Face Animation with Generative Adversarial Networks
• EasyCom: An Augmented Reality Dataset to Support Algorithms for Easy Communication in Noisy Environments
• AM-MobileNet1D: A Portable Model for Speaker Recognition
• SVFAP: Self-supervised Video Facial Affect Perceiver
• SpeakerNet: 1D Depth-wise Separable Convolutional Network for Text-Independent Speaker Recognition and Verification
• Semi-Supervised Contrastive Learning with Generalized Contrastive Loss and Its Application to Speaker Recognition
• SkipConvNet: Skip Convolutional Neural Network for Speech Dereverberation using Optimally Smoothed Spectral Mapping
• Multi-source Domain Adaptation for Text-independent Forensic Speaker Recognition
• BERTphone: Phonetically-Aware Encoder Representations for Utterance-Level Speaker and Language Recognition
• Leveraging ASR Pretrained Conformers for Speaker Verification through Transfer Learning and Knowledge Distillation
• The VoicePrivacy 2020 Challenge Evaluation Plan
• LAEO-Net++: revisiting people Looking At Each Other in videos
• iQIYI-VID: A Large Dataset for Multi-modal Person Identification
• On the Resilience of Biometric Authentication Systems against Random Inputs
• Attention Back-end for Automatic Speaker Verification with Multiple Enrollment Utterances
• C3-DINO: Joint Contrastive and Non-contrastive Self-Supervised Learning for Speaker Verification
• Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation
• Families In Wild Multimedia: A Multimodal Database for Recognizing Kinship
• FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Dataset
• Margin Matters: Towards More Discriminative Deep Neural Network Embeddings for Speaker Recognition
• The DKU-DukeECE Systems for VoxCeleb Speaker Recognition Challenge 2020
• EfficientTDNN: Efficient Architecture Search for Speaker Recognition
• One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing
• NeXt-TDNN: Modernizing Multi-Scale Temporal Convolution Backbone for Speaker Verification
• The VoxCeleb Speaker Recognition Challenge: A Retrospective
• A Comparative Study of Self-supervised Speech Representation Based Voice Conversion
• Linkage Based Face Clustering via Graph Convolution Network
• Defend Data Poisoning Attacks on Voice Authentication
• Multi-modal embeddings using multi-task learning for emotion recognition
• USTC-NELSLIP System Description for DIHARD-III Challenge
• Graph-based Label Propagation for Semi-Supervised Speaker Identification
• BUT System Description for DIHARD Speech Diarization Challenge 2019
• AVTENet: A Human-Cognition-Inspired Audio-Visual Transformer-Based Ensemble Network for Video Deepfake Detection
• Non-Contrastive Self-supervised Learning for Utterance-Level Information Extraction from Speech
• Cross-Lingual Speaker Verification with Domain-Balanced Hard Prototype Mining and Language-Dependent Score Normalization
• Expressive Neural Voice Cloning
• Few-Shot Speaker Identification Using Lightweight Prototypical Network with Feature Grouping and Interaction
• Combination of Deep Speaker Embeddings for Diarisation
• Advances in Online Audio-Visual Meeting Transcription
• Additive Margin SincNet for Speaker Recognition
• Large Margin Softmax Loss for Speaker Verification
• Utterance-level Aggregation For Speaker Recognition In The Wild
• Channel adversarial training for speaker verification and diarization
• Revealing Emotional Clusters in Speaker Embeddings: A Contrastive Learning Strategy for Speech Emotion Recognition
• Assessing the Generalization Gap of Learning-Based Speech Enhancement Systems in Noisy and Reverberant Environments
• Emotionless: Privacy-Preserving Speech Analysis for Voice Assistants
• Am I a Real or Fake Celebrity? Measuring Commercial Face Recognition Web APIs under Deepfake Impersonation Attack
• Deep Audio-Visual Learning: A Survey
• The IDLAB VoxCeleb Speaker Recognition Challenge 2020 System Description
• Speaker Anonymization with Distribution-Preserving X-Vector Generation for the VoicePrivacy Challenge 2020
• The effect of speech pathology on automatic speaker verification -- a large-scale study
• Contrastive Learning of Person-independent Representations for Facial Action Unit Detection
• Lightweight Speaker Verification for Online Identification of New Speakers with Short Segments
• Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wild
• Meta-Learning for Short Utterance Speaker Recognition with Imbalance Length Pairs
• AutoSpeech: Neural Architecture Search for Speaker Recognition
• CN-CELEB: a challenging Chinese speaker recognition dataset
• ResNeXt and Res2Net Structures for Speaker Verification
• Improving fairness in speaker verification via Group-adapted Fusion Network
• Adversarial speech for voice privacy protection from Personalized Speech generation
• Speaker attribution with voice profiles by graph-based semi-supervised learning
• Large-scale multilingual audio visual dubbing
• Few-shot Forgery Detection via Guided Adversarial Interpolation
• Text adaptation for speaker verification with speaker-text factorized embeddings
• MM-ALT: A Multimodal Automatic Lyric Transcription System
• Learning spectro-temporal representations of complex sounds with parameterized neural networks
• The JHU submission to VoxSRC-21: Track 3
• Speech Separation with Pretrained Frontend to Minimize Domain Mismatch
• VisualVoice: Audio-Visual Speech Separation with Cross-Modal Consistency
• End-to-End Zero-Shot Voice Conversion with Location-Variable Convolutions
• openFEAT: Improving Speaker Identification by Open-set Few-shot Embedding Adaptation with Transformer
• Optimizing Tandem Speaker Verification and Anti-Spoofing Systems
• Talking Head Generation with Audio and Speech Related Facial Action Units
• The DKU-DukeECE System for the Self-Supervision Speaker Verification Task of the 2021 VoxCeleb Speaker Recognition Challenge
• Centroid-based deep metric learning for speaker recognition
• Contrastive-mixup learning for improved speaker verification
• Voice-Face Cross-modal Matching and Retrieval: A Benchmark
• Exploiting Pre-Trained ASR Models for Alzheimer's Disease Recognition Through Spontaneous Speech
• Coding Speech through Vocal Tract Kinematics
• Speaker Recognition Based on Deep Learning: An Overview
• Neuralogram: A Deep Neural Network Based Representation for Audio Signals
• MAAS: Multi-modal Assignation for Active Speaker Detection
• CN-Celeb: multi-genre speaker recognition
• Training speaker recognition systems with limited data
• The ByteDance Speaker Diarization System for the VoxCeleb Speaker Recognition Challenge 2021
• Proposal-based Few-shot Sound Event Detection for Speech and Environmental Sounds with Perceivers
• Adversarial defense for automatic speaker verification by cascaded self-supervised learning models
• Deep Normalization for Speaker Vectors
• Time-Domain Speech Extraction with Spatial Information and Multi Speaker Conditioning Mechanism
• Zero-Shot Multi-Speaker Text-To-Speech with State-of-the-art Neural Speaker Embeddings
• One-Shot Identity-Preserving Portrait Reenactment
• X-TaSNet: Robust and Accurate Time-Domain Speaker Extraction Network
• A Comparison of Metric Learning Loss Functions for End-To-End Speaker Verification
• The xx205 System for the VoxCeleb Speaker Recognition Challenge 2020
• Towards Lightweight Speaker Verification via Adaptive Neural Network Quantization
• Cross-attention Inspired Selective State Space Models for Target Sound Extraction
• Complex-Valued Time-Frequency Self-Attention for Speech Dereverberation
• Fully Supervised Speaker Diarization
• Learning Speaker Embedding with Momentum Contrast
• Leveraging Visual Supervision for Array-based Active Speaker Detection and Localization
• SLMIA-SR: Speaker-Level Membership Inference Attacks against Speaker Recognition Systems
• Improved RawNet with Feature Map Scaling for Text-independent Speaker Verification using Raw Waveforms
• Cosine Scoring with Uncertainty for Neural Speaker Embedding
• Sparsely Overlapped Speech Training in the Time Domain: Joint Learning of Target Speech Separation and Personal VAD Benefits
• Semi-supervised acoustic modelling for five-lingual code-switched ASR using automatically-segmented soap opera speech
• MorphGAN: One-Shot Face Synthesis GAN for Detecting Recognition Bias
• HLT-NUS SUBMISSION FOR 2020 NIST Conversational Telephone Speech SRE
• Voice Cloning: a Multi-Speaker Text-to-Speech Synthesis Approach based on Transfer Learning
• Versatile Face Animator: Driving Arbitrary 3D Facial Avatar in RGBD Space
• Monocular Real-time Full Body Capture with Inter-part Correlations
• Voice-Indistinguishability: Protecting Voiceprint in Privacy-Preserving Speech Data Release
• A Survey on Speech Large Language Models for Understanding
• Static for Dynamic: Towards a Deeper Understanding of Dynamic Facial Expressions Using Static Expression Data
• VGGSound: A Large-scale Audio-Visual Dataset
• UWB-NTIS Speaker Diarization System for the DIHARD II 2019 Challenge
• Tackling the Score Shift in Cross-Lingual Speaker Verification by Exploiting Language Information
• Continuous Speech Separation Using Speaker Inventory for Long Multi-talker Recording
• LMD: A Learnable Mask Network to Detect Adversarial Examples for Speaker Verification
• $M^3$T: Multi-Modal Continuous Valence-Arousal Estimation in the Wild
• Detecting and analyzing missing citations to published scientific entities
• Bootstrap Equilibrium and Probabilistic Speaker Representation Learning for Self-supervised Speaker Verification
• Who said that?: Audio-visual speaker diarisation of real-world meetings
• Deep Speaker Embeddings for Far-Field Speaker Recognition on Short Utterances
• Task-agnostic Temporally Consistent Facial Video Editing
• Joint Speaker Counting, Speech Recognition, and Speaker Identification for Overlapped Speech of Any Number of Speakers
• ConVoice: Real-Time Zero-Shot Voice Style Transfer with Convolutional Network
• From Speaker Verification to Multispeaker Speech Synthesis, Deep Transfer with Feedback Constraint
• A Multi-View Approach To Audio-Visual Speaker Verification
• MP3net: coherent, minute-long music generation from raw audio with a simple convolutional GAN
• Diffusion-Based Adversarial Purification for Speaker Verification
• Adversarial Reweighting for Speaker Verification Fairness
• Self-supervised Speaker Recognition Training Using Human-Machine Dialogues
• Improving Speaker Identification for Shared Devices by Adapting Embeddings to Speaker Subsets
• On Cross-Corpus Generalization of Deep Learning Based Speech Enhancement
• Exploring Machine Speech Chain for Domain Adaptation and Few-Shot Speaker Adaptation
• Single-Channel Speech Separation with Auxiliary Speaker Embeddings
• Contrastive Unsupervised Learning for Audio Fingerprinting
• Unsupervised Audiovisual Synthesis via Exemplar Autoencoders
• N-HANS: Introducing the Augsburg Neuro-Holistic Audio-eNhancement System
• Learning from Videos with Deep Convolutional LSTM Networks
• A Study of Few-Shot Audio Classification
• ForgeryNet: A Versatile Benchmark for Comprehensive Forgery Analysis
• Talking-head Generation with Rhythmic Head Motion
• Exploring Deep Learning for Joint Audio-Visual Lip Biometrics
• Optimizing Multi-Taper Features for Deep Speaker Verification
• Uncovering the Visual Contribution in Audio-Visual Speech Recognition
• Transfer Learning from Monolingual ASR to Transcription-free Cross-lingual Voice Conversion
• Pairwise Discriminative Neural PLDA for Speaker Verification
• Deep Speaker Vector Normalization with Maximum Gaussianality Training
• DiDiSpeech: A Large Scale Mandarin Speech Corpus
• Revisiting Cross Modal Retrieval
• Speech Fusion to Face: Bridging the Gap Between Human's Vocal Characteristics and Facial Imaging
• Voting for the right answer: Adversarial defense for speaker verification
• FreeSVC: Towards Zero-shot Multilingual Singing Voice Conversion
• Sub-word Level Lip Reading With Visual Attention
• An Attribute-Aligned Strategy for Learning Speech Representation
• GMM-ResNext: Combining Generative and Discriminative Models for Speaker Verification
• An iterative framework for self-supervised deep speaker representation learning
• Domain-Invariant Speaker Vector Projection by Model-Agnostic Meta-Learning
• Nowhere to Hide: Cross-modal Identity Leakage between Biometrics and Devices
• Can Speaker Augmentation Improve Multi-Speaker End-to-End TTS?
• North America Bixby Speaker Diarization System for the VoxCeleb Speaker Recognition Challenge 2021
• The UPC Speaker Verification System Submitted to VoxCeleb Speaker Recognition Challenge 2020 (VoxSRC-20)
• Riggable 3D Face Reconstruction via In-Network Optimization
• Investigation of End-To-End Speaker-Attributed ASR for Continuous Multi-Talker Recordings
• Single Channel Far Field Feature Enhancement For Speaker Verification In The Wild
• Intra-class variation reduction of speaker representation in disentanglement framework
• Multi-Modal Emotion Detection with Transfer Learning
• Improved Meta-Learning Training for Speaker Verification
• Style Equalization: Unsupervised Learning of Controllable Generative Sequence Models
• Reference-Based Face Super-Resolution Using the Spatial Transformer
• A Memory Augmented Architecture for Continuous Speaker Identification in Meetings
• Personalized Face Modeling for Improved Face Reconstruction and Motion Retargeting
• Rep Works in Speaker Verification
• RAVSS: Robust Audio-Visual Speech Separation in Multi-Speaker Scenarios with Missing Visual Cues
• Multimodal Target Speech Separation with Voice and Face References
• A Study on Angular Based Embedding Learning for Text-independent Speaker Verification
• SpeechNAS: Towards Better Trade-off between Latency and Accuracy for Large-Scale Speaker Verification
• VidFace: A Full-Transformer Solver for Video FaceHallucination with Unaligned Tiny Snapshots
• End-to-End Speaker-Attributed ASR with Transformer
• Everything's Talkin': Pareidolia Face Reenactment
• Building Bilingual and Code-Switched Voice Conversion with Limited Training Data Using Embedding Consistency Loss
• SEEF-ALDR: A Speaker Embedding Enhancement Framework via Adversarial Learning based Disentangled Representation
• An Attention-Based Speaker Naming Method for Online Adaptation in Non-Fixed Scenarios
• Supervised Speaker Embedding De-Mixing in Two-Speaker Environment
• An end-to-end approach for the verification problem: learning the right distance
• HeightCeleb - an enrichment of VoxCeleb dataset with speaker height information
• Audio-Visual Kinship Verification
• Improving Embedding Extraction for Speaker Verification with Ladder Network
• The sound of my voice: speaker representation loss for target voice separation
• Structural sparsification for Far-field Speaker Recognition with GNA
• TEASEL: A Transformer-Based Speech-Prefixed Language Model
• Self-supervised Reflective Learning through Self-distillation and Online Clustering for Speaker Representation Learning
• PIE: Portrait Image Embedding for Semantic Control
• Speech2Video: Cross-Modal Distillation for Speech to Video Generation
• Text-Independent Speaker Verification with Dual Attention Network
• Beyond Neural-on-Neural Approaches to Speaker Gender Protection
• The HCCL Speaker Verification System for Far-Field Speaker Verification Challenge
• Minimum Bayes Risk Training for End-to-End Speaker-Attributed ASR
• Sub-band Convolutional Neural Networks for Small-footprint Spoken Term Classification
• ET-GAN: Cross-Language Emotion Transfer Based on Cycle-Consistent Generative Adversarial Networks
• Muse: Multi-modal target speaker extraction with visual cues
• A Unified Deep Speaker Embedding Framework for Mixed-Bandwidth Speech Data
• Speaker Diarization with Region Proposal Network
• Self-supervised Text-independent Speaker Verification using Prototypical Momentum Contrastive Learning
• Quantifying Source Speaker Leakage in One-to-One Voice Conversion
• Speaker Recognition with Random Digit Strings Using Uncertainty Normalized HMM-based i-vectors
• A Comparative Study of Modular and Joint Approaches for Speaker-Attributed ASR on Monaural Long-Form Audio
• Extreme-scale Talking-Face Video Upsampling with Audio-Visual Priors
• Serialized Multi-Layer Multi-Head Attention for Neural Speaker Embedding
• Robust Acoustic Domain Identification with its Application to Speaker Diarization
• Look Who's Talking: Active Speaker Detection in the Wild
• Speaker Characterization by means of Attention Pooling
• Fusion of Embeddings Networks for Robust Combination of Text Dependent and Independent Speaker Recognition
• Advancing Talking Head Generation: A Comprehensive Survey of Multi-Modal Methodologies, Datasets, Evaluation Metrics, and Loss Functions
• Poformer: A simple pooling transformer for speaker verification
• Self-Tuning Spectral Clustering for Speaker Diarization
• Sample-Independent Federated Learning Backdoor Attack in Speaker Recognition
• Deep generative LDA
• ShaneRun System Description to VoxCeleb Speaker Recognition Challenge 2020
• Graph-based Multi-View Fusion and Local Adaptation: Mitigating Within-Household Confusability for Speaker Identification
• DEAAN: Disentangled Embedding and Adversarial Adaptation Network for Robust Speaker Representation Learning
• Automatic Face Understanding: Recognizing Families in Photos
• The DKU System for the Speaker Recognition Task of the 2019 VOiCES from a Distance Challenge
• GaitSet: Cross-view Gait Recognition through Utilizing Gait as a Deep Set
• FineNet: Frame Interpolation and Enhancement for Face Video Deblurring
• A Study on Decoupled Probabilistic Linear Discriminant Analysis
• Temporal Dynamic Convolutional Neural Network for Text-Independent Speaker Verification and Phonemetic Analysis
• Diarization of Legal Proceedings. Identifying and Transcribing Judicial Speech from Recorded Court Audio
• USEV: Universal Speaker Extraction with Visual Cue
• Segment Aggregation for short utterances speaker verification using raw waveforms
• Neural Discriminant Analysis for Deep Speaker Embedding
• The HUAWEI Speaker Diarisation System for the VoxCeleb Speaker Diarisation Challenge
• Unsupervised Representation Learning for Speaker Recognition via Contrastive Equilibrium Learning
• Momentum Contrast Speaker Representation Learning
• HLT-NUS Submission for NIST 2019 Multimedia Speaker Recognition Evaluation
• Y-Vector: Multiscale Waveform Encoder for Speaker Embedding
• Compositional embedding models for speaker identification and diarization with simultaneous speech from 2+ speakers
• UIAI System for Short-Duration Speaker Verification Challenge 2020
• MIRNet: Learning multiple identities representations in overlapped speech
• Audio-visual Speaker Recognition with a Cross-modal Discriminative Network
• Mixture of Speaker-type PLDAs for Children's Speech Diarization
• Open-set Short Utterance Forensic Speaker Verification using Teacher-Student Network with Explicit Inductive Bias
• Cross-modal Speaker Verification and Recognition: A Multilingual Perspective
• Delving into VoxCeleb: environment invariant speaker recognition
• Mixture factorized auto-encoder for unsupervised hierarchical deep factorization of speech signal
• What does a network layer hear? Analyzing hidden representations of end-to-end ASR through speech synthesis
• Speaker Identification using EEG
• Partial AUC optimization based deep speaker embeddings with class-center learning for text-independent speaker verification
• The DKU-SMIIP System for NIST 2018 Speaker Recognition Evaluation
• Self-supervised pre-training with acoustic configurations for replay spoofing detection
• HI-MIA : A Far-field Text-Dependent Speaker Verification Database and the Baselines
• DropClass and DropAdapt: Dropping classes for deep speaker representation learning
• AlignNet: A Unifying Approach to Audio-Visual Alignment
• End-to-end losses based on speaker basis vectors and all-speaker hard negative mining for speaker verification
• Multi-Task Learning with High-Order Statistics for X-vector based Text-Independent Speaker Verification
• Speaker Verification By Partial AUC Optimization With Mahalanobis Distance Metric Learning
• Adaptive Margin Circle Loss for Speaker Verification
• Multi-task Metric Learning for Text-independent Speaker Verification
• JNR: Joint-based Neural Rig Representation for Compact 3D Face Modeling
• An Improved Deep Neural Network for Modeling Speaker Characteristics at Different Temporal Scales
• Cross-Identity Motion Transfer for Arbitrary Objects through Pose-Attentive Video Reassembling
• Exploring End-to-End Multi-channel ASR with Bias Information for Meeting Transcription
• Voice Aging with Audio-Visual Style Transfer
• Exploring Voice Conversion based Data Augmentation in Text-Dependent Speaker Verification
• Noise-tolerant Audio-visual Online Person Verification using an Attention-based Neural Network Fusion
• Self-Attentive Multi-Layer Aggregation with Feature Recalibration and Normalization for End-to-End Speaker Verification System
• Utterance Clustering Using Stereo Audio Channels
• Designing Neural Speaker Embeddings with Meta Learning
• Query Expansion System for the VoxCeleb Speaker Recognition Challenge 2020
• Estimating Uniqueness of I-Vector Representation of Human Voice
• Large-scale Speaker Retrieval on Random Speaker Variability Subspace
• PL-EESR: Perceptual Loss Based END-TO-END Robust Speaker Representation Extraction
• A Comprehensive Multi-scale Approach for Speech and Dynamics Synchrony in Talking Head Generation
• Xi-Vector Embedding for Speaker Recognition
• Cross-domain Adaptation with Discrepancy Minimization for Text-independent Forensic Speaker Verification
• Facial Keypoint Sequence Generation from Audio
• When Automatic Voice Disguise Meets Automatic Speaker Verification
• Squeezing value of cross-domain labels: a decoupled scoring approach for speaker verification
• Improving Text-Independent Speaker Verification with Auxiliary Speakers Using Graph
• AnyoneNet: Synchronized Speech and Talking Head Generation for Arbitrary Person
• Evaluation of Human and Machine Face Detection using a Novel Distinctive Human Appearance Dataset
• Non-native Speaker Verification for Spoken Language Assessment
• Sparse to Dense Motion Transfer for Face Image Animation
• Long-Term Conversation Analysis: Privacy-Utility Trade-off under Noise and Reverberation
• Enrollment-less training for personalized voice activity detection
• AE-StyleGAN: Improved Training of Style-Based Auto-Encoders
• Voice Biometrics Security: Extrapolating False Alarm Rate via Hierarchical Bayesian Modeling of Speaker Verification Scores
• Voice Passing : a Non-Binary Voice Gender Prediction System for evaluating Transgender voice transition
• Duality Temporal-channel-frequency Attention Enhanced Speaker Representation Learning
• X-Vectors with Multi-Scale Aggregation for Speaker Diarization
• Data Augmentation Methods for End-to-end Speech Recognition on Distant-Talk Scenarios
• Self-Supervised Frameworks for Speaker Verification via Bootstrapped Positive Sampling
• UniFaceGAN: A Unified Framework for Temporally Consistent Facial Video Editing
• An Exploration of ECAPA-TDNN and x-vector Speaker Representations in Zero-shot Multi-speaker TTS
• Hyperbolic Additive Margin Softmax with Hierarchical Information for Speaker Verification
• Single-Microphone Audio Point Source Discriminative Localization From Reverberation Late Tail Estimation
• EasyCall corpus: a dysarthric speech dataset
• Perceptual Gradient Networks
• Learning Metrics from Mean Teacher: A Supervised Learning Method for Improving the Generalization of Speaker Verification System
• Streaming Multi-talker Speech Recognition with Joint Speaker Identification
• Collaborative Feature Aggregation for Face Super-Resolution and Robust Re-Identification
• Bidirectional Multiscale Feature Aggregation for Speaker Verification
• Towards Robust Speaker Verification with Target Speaker Enhancement
• Harnessing Geometric Constraints from Emotion Labels to improve Face Verification
• Seeking the Shape of Sound: An Adaptive Framework for Learning Voice-Face Association
• Real-time RGBD-based Extended Body Pose Estimation
• Coupling a generative model with a discriminative learning framework for speaker verification
• Active Speakers in Context
• Speaker diarization with session-level speaker embedding refinement using graph neural networks
• Face and Voice Cross-modal Association with Learning Convex Feature Embedding
• Inclusive Speaker Verification with Adaptive thresholding
• Non-local convolutional neural networks (nlcnn) for speaker recognition
• How Far Are We from Robust Voice Conversion: A Survey
• An MAP Estimation for Between-Class Variance
• Generalized Operating Procedure for Deep Learning: an Unconstrained Optimal Design Perspective
• Multi Modal Adaptive Normalization for Audio to Video Generation
• MCSAE: Masked Cross Self-Attentive Encoding for Speaker Embedding
• Supervised attention for speaker recognition
• EML System Description for VoxCeleb Speaker Diarization Challenge 2020
• MuteSwap: Visual-informed Silent Video Identity Conversion
• Large-scale Multi-modal Person Identification in Real Unconstrained Environments
• Tongji University Undergraduate Team for the VoxCeleb Speaker Recognition Challenge2020
• Tongji University Team for the VoxCeleb Speaker Recognition Challenge 2020
• MoEVC: A Mixture-of-experts Voice Conversion System with Sparse Gating Mechanism for Accelerating Online Computation