Streaming Audio-Visual Speech Recognition with Alignment Regularization

Pingchuan Ma, Niko Moritz, Stavros Petridis +2

In this work, we propose a streaming AV-ASR system based on a hybrid connectionist temporal classification (CTC)/attention neural network architecture. The audio and the visual enc…