An Empirical Analysis of Deep Audio-Visual Models for Speech Recognition
arXiv:1812.09336
Abstract
In this project, we worked on speech recognition, specifically predicting individual words based on both the video frames and audio. Empowered by convolutional neural networks, the recent speech recognition and lip reading models are comparable to human level performance. We re-implemented and made derivations of the state-of-the-art model. Then, we conducted rich experiments including the effectiveness of attention mechanism, more accurate residual network as the backbone with pre-trained weights and the sensitivity of our model with respect to audio input with/without noise.
References in corpus (6)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- AMC: AutoML for Model Compression and Acceleration on Mobile Devices
- LipNet: End-to-End Sentence-level Lipreading
- 3D Convolutional Neural Networks for Cross Audio-Visual Matching Recognition
- Attention-based Audio-Visual Fusion for Robust Automatic Speech Recognition
- Single Image Super-resolution via a Lightweight Residual Convolutional Neural Network