Exploring the Encoding Layer and Loss Function in End-to-End Speaker and Language Recognition System

Abstract

In this paper, we explore the encoding/pooling layer and loss function in the end-to-end speaker and language recognition system. First, a unified and interpretable end-to-end system for both speaker and language recognition is developed. It accepts variable-length input and produces an utterance level result. In the end-to-end system, the encoding layer plays a role in aggregating the variable-length input sequence into an utterance level representation. Besides the basic temporal average pooling, we introduce a self-attentive pooling layer and a learnable dictionary encoding layer to get the utterance level representation. In terms of loss function for open-set speaker verification, to get more discriminative speaker embedding, center loss and angular softmax loss is introduced in the end-to-end system. Experimental results on Voxceleb and NIST LRE 07 datasets show that the performance of end-to-end learning system could be significantly improved by the proposed encoding layer and loss function.

arXiv commentAccepted for Speaker Odyssey 2018

References in corpus (3)

• Deep Speaker: an End-to-End Neural Speaker Embedding System
• SphereFace: Deep Hypersphere Embedding for Face Recognition
• Attention-Based Models for Text-Dependent Speaker Verification

Cited by in corpus (16)

• Utterance-level Aggregation For Speaker Recognition In The Wild
• AutoSpeech: Neural Architecture Search for Speaker Recognition
• Analysis of Length Normalization in End-to-End Speaker Verification System
• Intra-class variation reduction of speaker representation in disentanglement framework
• Deep Discriminative Feature Learning for Accent Recognition
• Speaker Recognition with Random Digit Strings Using Uncertainty Normalized HMM-based i-vectors
• Open-set Short Utterance Forensic Speaker Verification using Teacher-Student Network with Explicit Inductive Bias
• Neural Discriminant Analysis for Deep Speaker Embedding
• The DKU-Duke-Lenovo System Description for the Third DIHARD Speech Diarization Challenge
• Detection and Evaluation of human and machine generated speech in spoofing attacks on automatic speaker verification systems
• Adaptive Margin Circle Loss for Speaker Verification
• Cross-domain Adaptation with Discrepancy Minimization for Text-independent Forensic Speaker Verification
• Speaker Representation Learning using Global Context Guided Channel and Time-Frequency Transformations
• Bidirectional Multiscale Feature Aggregation for Speaker Verification
• Non-local convolutional neural networks (nlcnn) for speaker recognition
• Tongji University Undergraduate Team for the VoxCeleb Speaker Recognition Challenge2020