17 citations · 65 across the 38 of their papers we have counts for
13 papers · 1 filter
Sequence Transduction with Graph-based Supervision
Niko Moritz, Takaaki Hori, Shinji Watanabe +1
The recurrent neural network transducer (RNN-T) objective plays a major role in building today's best automatic speech recognition (ASR) systems for production. Similarly to the co…
Audio-Visual Scene-Aware Dialog and Reasoning using Audio-Visual Transformers with Joint Student-Teacher Learning
Ankit P. Shah, Shijie Geng, Peng Gao +5
In previous work, we have proposed the Audio-Visual Scene-Aware Dialog (AVSD) task, collected an AVSD dataset, developed AVSD technologies, and hosted an AVSD challenge track at bo…
Advancing Momentum Pseudo-Labeling with Conformer and Initialization Strategy
Yosuke Higuchi, Niko Moritz, Jonathan Le Roux +1
Pseudo-labeling (PL), a semi-supervised learning (SSL) method where a seed model performs self-training using pseudo-labels generated from untranscribed speech, has been shown to e…
The Cocktail Fork Problem: Three-Stem Audio Separation for Real-World Soundtracks
Darius Petermann, Gordon Wichern, Zhong-Qiu Wang +1
The cocktail party problem aims at isolating any source of interest within a complex acoustic scene, and has long inspired audio source separation research. Recent efforts have mai…
Leveraging Low-Distortion Target Estimates for Improved Speech Enhancement
Zhong-Qiu Wang, Gordon Wichern, Jonathan Le Roux
A promising approach for multi-microphone speech separation involves two deep neural networks (DNN), where the predicted target speech from the first DNN is used to compute signal…
Visual Scene Graphs for Audio Source Separation
Moitreya Chatterjee, Jonathan Le Roux, Narendra Ahuja +1
State-of-the-art approaches for visually-guided audio source separation typically assume sources that have characteristic sounds, such as musical instruments. These approaches ofte…
Convolutive Prediction for Reverberant Speech Separation
Zhong-Qiu Wang, Gordon Wichern, Jonathan Le Roux
We investigate the effectiveness of convolutive prediction, a novel formulation of linear prediction for speech dereverberation, for speaker separation in reverberant conditions. T…
Optimizing Latency for Online Video CaptioningUsing Audio-Visual Transformers
Chiori Hori, Takaaki Hori, Jonathan Le Roux
Video captioning is an essential technology to understand scenes and describe events in natural language. To apply it to real-time monitoring, a system needs not only to describe e…
Convolutive Prediction for Monaural Speech Dereverberation and Noisy-Reverberant Speaker Separation
Zhong-Qiu Wang, Gordon Wichern, Jonathan Le Roux
A promising approach for speech dereverberation is based on supervised learning, where a deep neural network (DNN) is trained to predict the direct sound from noisy-reverberant spe…
Dual Causal/Non-Causal Self-Attention for Streaming End-to-End Speech Recognition
Niko Moritz, Takaaki Hori, Jonathan Le Roux
Attention-based end-to-end automatic speech recognition (ASR) systems have recently demonstrated state-of-the-art results for numerous tasks. However, the application of self-atten…
Momentum Pseudo-Labeling for Semi-Supervised Speech Recognition
Yosuke Higuchi, Niko Moritz, Jonathan Le Roux +1
Pseudo-labeling (PL) has been shown to be effective in semi-supervised automatic speech recognition (ASR), where a base model is self-trained with pseudo-labels generated from unla…
Advanced Long-context End-to-end Speech Recognition Using Context-expanded Transformers
Takaaki Hori, Niko Moritz, Chiori Hori +1
This paper addresses end-to-end automatic speech recognition (ASR) for long audio recordings such as lecture and conversational speeches. Most end-to-end ASR models are designed to…
Capturing Multi-Resolution Context by Dilated Self-Attention
Niko Moritz, Takaaki Hori, Jonathan Le Roux
Self-attention has become an important and widely used neural network component that helped to establish new state-of-the-art results for various applications, such as machine tran…