Dense Prediction on Sequences with Time-Dilated Convolutions for Speech Recognition

Abstract

In computer vision pixelwise dense prediction is the task of predicting a label for each pixel in the image. Convolutional neural networks achieve good performance on this task, while being computationally efficient. In this paper we carry these ideas over to the problem of assigning a sequence of labels to a set of speech frames, a task commonly known as framewise classification. We show that dense prediction view of framewise classification offers several advantages and insights, including computational efficiency and the ability to apply batch normalization. When doing dense prediction we pay specific attention to strided pooling in time and introduce an asymmetric dilated convolution, called time-dilated convolution, that allows for efficient and elegant implementation of pooling in time. We show results using time-dilated convolutions in a very deep VGG-style CNN with batch normalization on the Hub5 Switchboard-2000 benchmark task. With a big n-gram language model, we achieve 7.7% WER which is the best single model single-pass performance reported so far.

Appeared at NIPS 2016 End-to-end Learning for Speech and Audio Processing Workshop

References in corpus (4)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Achieving Human Parity in Conversational Speech Recognition
• Wav2Letter: an End-to-End ConvNet-based Speech Recognition System
• Highly Efficient Forward and Backward Propagation of Convolutional Neural Networks for Pixelwise Classification

Cited by in corpus (13)

• Deep Learning for Audio Signal Processing
• Recent Advances in Convolutional Neural Networks
• Big-Little Net: An Efficient Multi-Scale Feature Representation for Visual and Speech Recognition
• Exploring Neural Transducers for End-to-End Speech Recognition
• Gram-CTC: Automatic Unit Selection and Target Decomposition for Sequence Labelling
• Score and Lyrics-Free Singing Voice Generation
• Reducing Bias in Production Speech Models
• A Multiversion Programming Inspired Approach to Detecting Audio Adversarial Examples
• Future Data Helps Training: Modeling Future Contexts for Session-based Recommendation
• Dilated Convolution with Dilated GRU for Music Source Separation
• A Simple Convolutional Generative Network for Next Item Recommendation
• Recent Progresses in Deep Learning based Acoustic Models (Updated)
• Multi-Frame Cross-Entropy Training for Convolutional Neural Networks in Speech Recognition