Multi-Task Network for Noise-Robust Keyword Spotting and Speaker Verification using CTC-based Soft VAD and Global Query Attention
arXiv:2005.03867
Abstract
Keyword spotting (KWS) and speaker verification (SV) have been studied independently although it is known that acoustic and speaker domains are complementary. In this paper, we propose a multi-task network that performs KWS and SV simultaneously to fully utilize the interrelated domain information. The multi-task network tightly combines sub-networks aiming at performance improvement in challenging conditions such as noisy environments, open-vocabulary KWS, and short-duration SV, by introducing novel techniques of connectionist temporal classification (CTC)-based soft voice activity detection (VAD) and global query attention. Frame-level acoustic and speaker information is integrated with phonetically originated weights so that forms a word-level global representation. Then it is used for the aggregation of feature vectors to generate discriminative embeddings. Our proposed approach shows 4.06% and 26.71% relative improvements in equal error rate (EER) compared to the baselines for both tasks. We also present a visualization example and results of ablation experiments.
Accepted to Interspeech 2020
References in corpus (6)
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition
- Attentive Statistics Pooling for Deep Speaker Embedding
- Deep Speaker: an End-to-End Neural Speaker Embedding System
- Improving Multi-Scale Aggregation Using Feature Pyramid Module for Robust Speaker Verification of Variable-Duration Utterances
- Spatial Pyramid Encoding with Convex Length Normalization for Text-Independent Speaker Verification