Tune-In: Training Under Negative Environments with Interference for Attention Networks Simulating Cocktail Party Effect
arXiv:2103.01461
Abstract
We study the cocktail party problem and propose a novel attention network called Tune-In, abbreviated for training under negative environments with interference. It firstly learns two separate spaces of speaker-knowledge and speech-stimuli based on a shared feature space, where a new block structure is designed as the building block for all spaces, and then cooperatively solves different tasks. Between the two spaces, information is cast towards each other via a novel cross- and dual-attention mechanism, mimicking the bottom-up and top-down processes of a human's cocktail party effect. It turns out that substantially discriminative and generalizable speaker representations can be learnt in severely interfered conditions via our self-supervised training. The experimental results verify this seeming paradox. The learnt speaker embedding has superior discriminative power than a standard speaker verification method; meanwhile, Tune-In achieves remarkably better speech separation performances in terms of SI-SNRi and SDRi consistently in all test modes, and especially at lower memory and computational consumption, than state-of-the-art benchmark systems.
Accepted in AAAI 2021
References in corpus (7)
- An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
- Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation
- Deep attractor network for single-microphone speaker separation
- Light Gated Recurrent Units for Speech Recognition
- LibriMix: An Open-Source Dataset for Generalizable Speech Separation
- Dual-Path Transformer Network: Direct Context-Aware Modeling for End-to-End Monaural Speech Separation
- WildMix Dataset and Spectro-Temporal Transformer Model for Monoaural Audio Source Separation