Noise-Robust AV-ASR Using Visual Features Both in the Whisper Encoder and Decoder

Zhengyang Li, Thomas Graave, Björn Möller +3

In audiovisual automatic speech recognition (AV-ASR) systems, information fusion of visual features in a pre-trained ASR has been proven as a promising method to improve noise robu…