Integrating Frequency Translational Invariance in TDNNs and Frequency Positional Information in 2D ResNets to Enhance Speaker Verification

Abstract

This paper describes the IDLab submission for the text-independent task of the Short-duration Speaker Verification Challenge 2021 (SdSVC-21). This speaker verification competition focuses on short duration test recordings and cross-lingual trials, along with the constraint of limited availability of in-domain DeepMine Farsi training data. Currently, both Time Delay Neural Networks (TDNNs) and ResNets achieve state-of-the-art results in speaker verification. These architectures are structurally very different and the construction of hybrid networks looks a promising way forward. We introduce a 2D convolutional stem in a strong ECAPA-TDNN baseline to transfer some of the strong characteristics of a ResNet based model to this hybrid CNN-TDNN architecture. Similarly, we incorporate absolute frequency positional encodings in an SE-ResNet34 architecture. These learnable feature map biases along the frequency axis offer this architecture a straightforward way to exploit frequency positional information. We also propose a frequency-wise variant of Squeeze-Excitation (SE) which better preserves frequency-specific information when rescaling the feature maps. Both modified architectures significantly outperform their corresponding baseline on the SdSVC-21 evaluation data and the original VoxCeleb1 test set. A four system fusion containing the two improved architectures achieved a third place in the final SdSVC-21 Task 2 ranking.

arXiv commentproceedings of INTERSPEECH 2021

References in corpus (15)

• Adam: A Method for Stochastic Optimization
• SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
• Res2Net: A New Multi-scale Backbone Architecture
• Convolutional Sequence to Sequence Learning
• VoxCeleb2: Deep Speaker Recognition
• Squeeze-and-Excitation Networks
• VoxCeleb: a large-scale speaker identification dataset
• ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification
• The BOSARIS Toolkit: Theory, Algorithms and Code for Surviving the New DCF
• The IDLAB VoxSRC-20 Submission: Large Margin Fine-Tuning and Quality-Aware Score Calibration in DNN Based Speaker Verification
• VoxSRC 2019: The first VoxCeleb Speaker Recognition Challenge
• Position, Padding and Predictions: A Deeper Look at Position Information in CNNs
• Cross-Lingual Speaker Verification with Domain-Balanced Hard Prototype Mining and Language-Dependent Score Normalization
• The IDLAB VoxCeleb Speaker Recognition Challenge 2020 System Description
• Short-duration Speaker Verification (SdSV) Challenge 2021: the Challenge Evaluation Plan

Cited by in corpus (5)

• Representation Selective Self-distillation and wav2vec 2.0 Feature Exploration for Spoof-aware Speaker Verification
• Tackling the Score Shift in Cross-Lingual Speaker Verification by Exploiting Language Information
• Beijing ZKJ-NPU Speaker Verification System for VoxCeleb Speaker Recognition Challenge 2021
• GMM-ResNext: Combining Generative and Discriminative Models for Speaker Verification
• Duality Temporal-channel-frequency Attention Enhanced Speaker Representation Learning