15 citations · 30 across the 14 of their papers we have counts for
14 papers
Multi-Channel Differential ASR for Robust Wearer Speech Recognition on Smart Glasses
Yufeng Yang, Yiteng Huang, Yong Xu +9
With the growing adoption of wearable devices such as smart glasses for AI assistants, wearer speech recognition (WSR) is becoming increasingly critical to next-generation human-co…
On Robustness to Missing Video for Audiovisual Speech Recognition
Oscar Chang, Otavio Braga, Hank Liao +2
It has been shown that learning audiovisual features can lead to improved speech recognition performance over audio-only features, especially for noisy speech. However, in many com…
Revisiting the Entropy Semiring for Neural Speech Recognition
Oscar Chang, Dongseong Hwang, Olivier Siohan
In streaming settings, speech recognition models have to map sub-sequences of speech to text before the full audio stream becomes available. However, since alignment information be…
Audio-visual fine-tuning of audio-only ASR models
Avner May, Dmitriy Serdyuk, Ankit Parag Shah +2
Audio-visual automatic speech recognition (AV-ASR) models are very effective at reducing word error rates on noisy speech, but require large amounts of transcribed AV training data…
Cascaded encoders for fine-tuning ASR models on overlapped speech
Richard Rose, Oscar Chang, Olivier Siohan
Multi-talker speech recognition (MT-ASR) has been shown to improve ASR performance on speech containing overlapping utterances from more than one speaker. Multi-talker models have…
Conformers are All You Need for Visual Speech Recognition
Oscar Chang, Hank Liao, Dmitriy Serdyuk +2
Visual speech recognition models extract visual features in a hierarchical manner. At the lower level, there is a visual front-end with a limited temporal receptive field that proc…
A Closer Look at Audio-Visual Multi-Person Speech Recognition and Active Speaker Selection
Otavio Braga, Olivier Siohan
Audio-visual automatic speech recognition is a promising approach to robust ASR under noisy conditions. However, up until recently it had been traditionally studied in isolation as…
End-to-End Multi-Person Audio/Visual Automatic Speech Recognition
Otavio Braga, Takaki Makino, Olivier Siohan +1
Traditionally, audio-visual automatic speech recognition has been studied under the assumption that the speaking face on the visual signal is the face matching the audio. However,…
Best of Both Worlds: Multi-task Audio-Visual Automatic Speech Recognition and Active Speaker Detection
Otavio Braga, Olivier Siohan
Under noisy conditions, automatic speech recognition (ASR) can greatly benefit from the addition of visual signals coming from a video of the speaker's face. However, when multiple…
End-to-end multi-talker audio-visual ASR using an active speaker attention module
Richard Rose, Olivier Siohan
This paper presents a new approach for end-to-end audio-visual multi-talker speech recognition. The approach, referred to here as the visual context attention model (VCAM), is impo…
Transformer-Based Video Front-Ends for Audio-Visual Speech Recognition for Single and Multi-Person Video
Dmitriy Serdyuk, Otavio Braga, Olivier Siohan
Audio-visual automatic speech recognition (AV-ASR) extends speech recognition by introducing the video modality as an additional source of information. In this work, the informatio…
Audio-Visual Speech Recognition is Worth 32328 Voxels
Dmitriy Serdyuk, Otavio Braga, Olivier Siohan
Audio-visual automatic speech recognition (AV-ASR) introduces the video modality into the speech recognition process, often by relying on information conveyed by the motion of the…
Bridging the gap between streaming and non-streaming ASR systems bydistilling ensembles of CTC and RNN-T models
Thibault Doutre, Wei Han, Chung-Cheng Chiu +3
Streaming end-to-end automatic speech recognition (ASR) systems are widely used in everyday applications that require transcribing speech to text in real-time. Their minimal latenc…
Recurrent Neural Network Transducer for Audio-Visual Speech Recognition
Takaki Makino, Hank Liao, Yannis Assael +4
This work presents a large-scale audio-visual speech recognition system based on a recurrent neural network transducer (RNN-T) architecture. To support the development of such a sy…