TransMask: A Compact and Fast Speech Separation Model Based on Transformer

Abstract

Speech separation is an important problem in speech processing, which targets to separate and generate clean speech from a mixed audio containing speech from different speakers. Empowered by the deep learning technologies over sequence-to-sequence domain, recent neural speech separation models are now capable of generating highly clean speech audios. To make these models more practical by reducing the model size and inference time while maintaining high separation quality, we propose a new transformer-based speech separation approach, called TransMask. By fully un-leashing the power of self-attention on long-term dependency exception, we demonstrate the size of TransMask is more than 60% smaller and the inference is more than 2 times faster than state-of-the-art solutions. TransMask fully utilizes the parallelism during inference, and achieves nearly linear inference time within reasonable input audio lengths. It also outperforms existing solutions on output speech audio quality, achieving SDR above 16 over Librimix benchmark.

arXiv commentAccepted in ICASSP2021

References in corpus (11)

• Gaussian Error Linear Units (GELUs)
• Longformer: The Long-Document Transformer
• Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation
• Generating Long Sequences with Sparse Transformers
• Transformer-based Acoustic Modeling for Hybrid Speech Recognition
• LibriMix: An Open-Source Dataset for Generalizable Speech Separation
• Transformers with convolutional context for ASR
• R-Transformer: Recurrent Neural Network Enhanced Transformer
• Dual-Path Transformer Network: Direct Context-Aware Modeling for End-to-End Monaural Speech Separation
• Continuous Speech Separation with Conformer
• Attention is All You Need in Speech Separation