Learning Multiscale Features Directly From Waveforms

Abstract

Deep learning has dramatically improved the performance of speech recognition systems through learning hierarchies of features optimized for the task at hand. However, true end-to-end learning, where features are learned directly from waveforms, has only recently reached the performance of hand-tailored representations based on the Fourier transform. In this paper, we detail an approach to use convolutional filters to push past the inherent tradeoff of temporal and frequency resolution that exists for spectral representations. At increased computational cost, we show that increasing temporal resolution via reduced stride and increasing frequency resolution via additional filters delivers significant performance improvements. Further, we find more efficient representations by simultaneously learning at multiple scales, leading to an overall decrease in word error rate on a difficult internal speech test set by 20.7% relative to networks with the same number of parameters trained on spectrograms.

arXiv comment"fix typo in the title"

References in corpus (2)

• Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
• Going Deeper with Convolutions

Cited by in corpus (11)

• GANSynth: Adversarial Neural Audio Synthesis
• CUHK & ETHZ & SIAT Submission to ActivityNet Challenge 2016
• Rethinking CNN Models for Audio Classification
• End-to-end learning for music audio tagging at scale
• Temporal Segment Networks for Action Recognition in Videos
• Adversarial Generation of Time-Frequency Features with application in audio synthesis
• Reducing Bias in Production Speech Models
• Neuralogram: A Deep Neural Network Based Representation for Audio Signals
• J-Net: Randomly weighted U-Net for audio source separation
• Multi-scale Embedded CNN for Music Tagging (MsE-CNN)
• Cross-scale Attention Model for Acoustic Event Classification