127 citations · 274 across the 15 of their papers we have counts for
21 papers
Speech inpainting: Context-based speech synthesis guided by video
Juan F. Montesinos, Daniel Michelsanti, Gloria Haro +2
Audio and visual modalities are inherently connected in speech signals: lip movements and facial expressions are correlated with speech sounds. This motivates studies that incorpor…
Filterbank Learning for Noise-Robust Small-Footprint Keyword Spotting
Iván López-Espejo, Ram C. M. C. Shekar, Zheng-Hua Tan +2
In the context of keyword spotting (KWS), the replacement of handcrafted speech features by learnable features has not yielded superior KWS performance. In this study, we demonstra…
Minimum Processing Near-end Listening Enhancement
Andreas Jonas Fuglsig, Jesper Jensen, Zheng-Hua Tan +3
The intelligibility and quality of speech from a mobile phone or public announcement system are often affected by background noise in the listening environment. By pre-processing t…
Deep Spoken Keyword Spotting: An Overview
Iván López-Espejo, Zheng-Hua Tan, John Hansen +1
Spoken keyword spotting (KWS) deals with the identification of keywords in audio streams and has become a fast-growing technology thanks to the paradigm shift introduced by deep le…
Joint Far- and Near-End Speech Intelligibility Enhancement based on the Approximated Speech Intelligibility Index
Andreas Jonas Fuglsig, Jan Østergaard, Jesper Jensen +3
This paper considers speech enhancement of signals picked up in one noisy environment which must be presented to a listener in another noisy environment. Recently, it has been show…
On TasNet for Low-Latency Single-Speaker Speech Enhancement
Morten Kolbæk, Zheng-Hua Tan, Søren Holdt Jensen +1
In recent years, speech processing algorithms have seen tremendous progress primarily due to the deep learning renaissance. This is especially true for speech separation where the…
Audio-Visual Speech Inpainting with Deep Learning
Giovanni Morrone, Daniel Michelsanti, Zheng-Hua Tan +1
In this paper, we present a deep-learning-based framework for audio-visual speech inpainting, i.e., the task of restoring the missing parts of an acoustic speech signal from reliab…
An Overview of Deep-Learning-Based Audio-Visual Speech Enhancement and Separation
Daniel Michelsanti, Zheng-Hua Tan, Shi-Xiong Zhang +4
Speech enhancement and speech separation are two related tasks, whose purpose is to extract either one or more target speech signals, respectively, from a mixture of sounds generat…
Exploring Filterbank Learning for Keyword Spotting
Iván López-Espejo, Zheng-Hua Tan, Jesper Jensen
Despite their great performance over the years, handcrafted speech features are not necessarily optimal for any particular speech application. Consequently, with greater or lesser…
Vocoder-Based Speech Synthesis from Silent Videos
Daniel Michelsanti, Olga Slizovskaia, Gloria Haro +3
Both acoustic and visual information influence human perception of speech. For this reason, the lack of audio in a video sequence determines an extremely low speech intelligibility…
On Loss Functions for Supervised Monaural Time-Domain Speech Enhancement
Morten Kolbæk, Zheng-Hua Tan, Søren Holdt Jensen +1
Many deep learning-based speech enhancement algorithms are designed to minimize the mean-square error (MSE) in some transform domain between a predicted and a target speech signal.…
Keyword Spotting for Hearing Assistive Devices Robust to External Speakers
Iván López-Espejo, Zheng-Hua Tan, Jesper Jensen
Keyword spotting (KWS) is experiencing an upswing due to the pervasiveness of small electronic devices that allow interaction with them via speech. Often, KWS systems are speaker-i…
Deep-Learning-Based Audio-Visual Speech Enhancement in Presence of Lombard Effect
Daniel Michelsanti, Zheng-Hua Tan, Sigurdur Sigurdsson +1
When speaking in presence of background noise, humans reflexively change their way of speaking in order to improve the intelligibility of their speech. This reflex is known as Lomb…
Effects of Lombard Reflex on the Performance of Deep-Learning-Based Audio-Visual Speech Enhancement Systems
Daniel Michelsanti, Zheng-Hua Tan, Sigurdur Sigurdsson +1
Humans tend to change their way of speaking when they are immersed in a noisy environment, a reflex known as Lombard effect. Current speech enhancement systems based on deep learni…
On Training Targets and Objective Functions for Deep-Learning-Based Audio-Visual Speech Enhancement
Daniel Michelsanti, Zheng-Hua Tan, Sigurdur Sigurdsson +1
Audio-visual speech enhancement (AV-SE) is the task of improving speech quality and intelligibility in a noisy environment using audio and visual information from a talker. Recentl…
On the Relationship Between Short-Time Objective Intelligibility and Short-Time Spectral-Amplitude Mean-Square Error for Speech Enhancement
Morten Kolbæk, Zheng-Hua Tan, Jesper Jensen
The majority of deep neural network (DNN) based speech enhancement algorithms rely on the mean-square error (MSE) criterion of short-time spectral amplitudes (STSA), which has no a…
Information Loss in the Human Auditory System
Mohsen Zareian Jahromi, Adel Zahedi, Jesper Jensen +1
From the eardrum to the auditory cortex, where acoustic stimuli are decoded, there are several stages of auditory processing and transmission where information may potentially get…
Monaural Speech Enhancement using Deep Neural Networks by Maximizing a Short-Time Objective Intelligibility Measure
Morten Kolbæk, Zheng-Hua Tan, Jesper Jensen
In this paper we propose a Deep Neural Network (DNN) based Speech Enhancement (SE) system that is designed to maximize an approximation of the Short-Time Objective Intelligibility…
Joint Separation and Denoising of Noisy Multi-talker Speech using Recurrent Neural Networks and Permutation Invariant Training
Morten Kolbæk, Dong Yu, Zheng-Hua Tan +1
In this paper we propose to use utterance-level Permutation Invariant Training (uPIT) for speaker independent multi-talker speech separation and denoising, simultaneously. Specific…
Multi-talker Speech Separation with Utterance-level Permutation Invariant Training of Deep Recurrent Neural Networks
Morten Kolbæk, Dong Yu, Zheng-Hua Tan +1
In this paper we propose the utterance-level Permutation Invariant Training (uPIT) technique. uPIT is a practically applicable, end-to-end, deep learning based solution for speaker…
Permutation Invariant Training of Deep Models for Speaker-Independent Multi-talker Speech Separation
Dong Yu, Morten Kolbæk, Zheng-Hua Tan +1
We propose a novel deep learning model, which supports permutation invariant training (PIT), for speaker independent multi-talker speech separation, commonly known as the cocktail-…