On the efficient representation and execution of deep acoustic models

Abstract

In this paper we present a simple and computationally efficient quantization scheme that enables us to reduce the resolution of the parameters of a neural network from 32-bit floating point values to 8-bit integer values. The proposed quantization scheme leads to significant memory savings and enables the use of optimized hardware instructions for integer arithmetic, thus significantly reducing the cost of inference. Finally, we propose a "quantization aware" training process that applies the proposed scheme during network training and find that it allows us to recover most of the loss in accuracy introduced by quantization. We validate the proposed techniques by applying them to a long short-term memory-based acoustic model on an open-ended large vocabulary speech recognition task.

arXiv commentAccepted conference paper: "The Annual Conference of the International Speech Communication Association (Interspeech), 2016"

References in corpus (9)

• Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
• BinaryConnect: Training Deep Neural Networks with binary weights during propagations
• Going Deeper with Convolutions
• Deep Learning with Limited Numerical Precision
• Long Short-Term Memory Based Recurrent Neural Network Architectures for Large Vocabulary Speech Recognition
• Bitwise Neural Networks
• Resiliency of Deep Neural Networks under Quantization
• Quantized Convolutional Neural Networks for Mobile Devices
• Personalized Speech recognition on mobile devices

Cited by in corpus (3)

• Edge Intelligence: Architectures, Challenges, and Applications
• VoiceFilter-Lite: Streaming Targeted Voice Separation for On-Device Speech Recognition
• Recent Progresses in Deep Learning based Acoustic Models (Updated)