Expoiting Syntactic Structure for Language Modeling

Abstract

The paper presents a language model that develops syntactic structure and uses it to extract meaningful information from the word history, thus enabling the use of long distance dependencies. The model assigns probability to every joint sequence of words--binary-parse-structure with headword annotation and operates in a left-to-right manner --- therefore usable for automatic speech recognition. The model, its probabilistic parameterization, and a set of experiments meant to evaluate its predictive power are presented; an improvement over standard trigram modeling is achieved.

changed ACM-class membership and buggy author names

Cited by in corpus (7)

• A Classification Approach to Word Prediction
• Structured Language Modeling for Speech Recognition
• Robust Probabilistic Predictive Syntactic Processing
• PaLM: A Hybrid Parser and Language Model
• How Do Neural Sequence Models Generalize? Local and Global Context Cues for Out-of-Distribution Prediction
• Probabilistic top-down parsing and language modeling
• A Bit of Progress in Language Modeling