Model Interpolation with Trans-dimensional Random Field Language Models for Speech Recognition

Abstract

The dominant language models (LMs) such as n-gram and neural network (NN) models represent sentence probabilities in terms of conditionals. In contrast, a new trans-dimensional random field (TRF) LM has been recently introduced to show superior performances, where the whole sentence is modeled as a random field. In this paper, we examine how the TRF models can be interpolated with the NN models, and obtain 12.1\% and 17.9\% relative error rate reductions over 6-gram LMs for English and Chinese speech recognition respectively through log-linear combination.

three pages, 2 experiment result tables, reporting the WERs on an Englisth dateset and a Chinese dataset

References in corpus (2)

• Recurrent Neural Network Regularization
• Feedforward Sequential Memory Networks: A New Structure to Learn Long-term Dependency

Cited by in corpus (2)

• Energy-Based Models with Applications to Speech and Language Processing
• Integrating Discrete and Neural Features via Mixed-feature Trans-dimensional Random Field Language Models